Reinterpret ratings

Have any bright ideas? Share and discuss them with the community

Moderators: Suggestions Team, Global Moderators

Forum rules
Please read the Community Guidelines before posting.

And don't forget to search for previously suggested ideas first!

Reinterpret ratings

Postby sundance123 on Sun Dec 02, 2012 4:40 pm

Concise description:

Amend the ratings description to reflect the reality of how users rate each other

Specifics/Details:

The current ratings system of

1 = Bad, 2 = Below Average, 3 = Average, 4 = Above Average, 5 = Excellent.

is nonsense - most people, IMO, will agree that 90% of users rate based on a different scale - something like

1 = Foed, 2 = this person is stupid and I hate them, 3 = this person is stupid, 4 = needs to learn a thing or two, 5 = I would play again with person.

*the exact wording to be worked out later.

The description of ratings could be easily changed and I am sure if a consensus view could be achieved on the description then most people who claim to have stuck rigidly to the current system would be happy to amend their ratings.

How this will benefit the site and/or other comments:

A ratings system that reflects users opinions would clearly benefit users who use the system and probably lead to a decrease in whinging C&A reports.

The current average rating on the site is 4.7 not 3.0 as many people are lead to believe.

89% of user have an rating of 4 or above.
User avatar
Major sundance123
 
Posts: 495
Joined: Fri Oct 30, 2009 6:29 pm
Medals: 69
Standard Achievement (4) Doubles Achievement (2) Triples Achievement (1) Quadruples Achievement (2) Terminator Achievement (2)
Assassin Achievement (2) Manual Troops Achievement (2) Freestyle Achievement (3) Nuclear Spoils Achievement (3) Fog of War Achievement (4)
Trench Warfare Achievement (2) Speed Achievement (2) Teammate Achievement (2) Random Map Achievement (3) Cross-Map Achievement (4)
Beta Map Achievement (1) Ratings Achievement (3) Tournament Achievement (15) General Achievement (11) Challenge Achievement (1)

Re: Reinterpret ratings

Postby sundance123 on Sun Dec 02, 2012 4:43 pm

sorry for the double post. I dont havae a photobucket account so I wont upload my frequency graph of ratings but here is the data as of today if anybody wants use it:
show: DATA
User avatar
Major sundance123
 
Posts: 495
Joined: Fri Oct 30, 2009 6:29 pm
Medals: 69
Standard Achievement (4) Doubles Achievement (2) Triples Achievement (1) Quadruples Achievement (2) Terminator Achievement (2)
Assassin Achievement (2) Manual Troops Achievement (2) Freestyle Achievement (3) Nuclear Spoils Achievement (3) Fog of War Achievement (4)
Trench Warfare Achievement (2) Speed Achievement (2) Teammate Achievement (2) Random Map Achievement (3) Cross-Map Achievement (4)
Beta Map Achievement (1) Ratings Achievement (3) Tournament Achievement (15) General Achievement (11) Challenge Achievement (1)

Re: Reinterpret ratings

Postby Little Witt on Mon Dec 03, 2012 12:13 am

I support! I think the current rating system is all messed up, and is in need of dire changes.
User avatar
Sergeant Little Witt
 
Posts: 539
Joined: Mon Mar 02, 2009 12:03 am
Location: in the USA
Medals: 51
Standard Achievement (4) Doubles Achievement (2) Triples Achievement (2) Quadruples Achievement (2) Terminator Achievement (2)
Assassin Achievement (3) Manual Troops Achievement (3) Freestyle Achievement (4) Nuclear Spoils Achievement (2) Fog of War Achievement (4)
Trench Warfare Achievement (1) Speed Achievement (3) Teammate Achievement (2) Cross-Map Achievement (3) Ratings Achievement (1)
Tournament Achievement (8) General Achievement (2) Tournament Contribution (3)

Re: Reinterpret ratings

Postby sundance123 on Tue Dec 04, 2012 4:27 am

Little Witt wrote:I support! I think the current rating system is all messed up, and is in need of dire changes.


You are right about that. Community doesn't give a toss though.
User avatar
Major sundance123
 
Posts: 495
Joined: Fri Oct 30, 2009 6:29 pm
Medals: 69
Standard Achievement (4) Doubles Achievement (2) Triples Achievement (1) Quadruples Achievement (2) Terminator Achievement (2)
Assassin Achievement (2) Manual Troops Achievement (2) Freestyle Achievement (3) Nuclear Spoils Achievement (3) Fog of War Achievement (4)
Trench Warfare Achievement (2) Speed Achievement (2) Teammate Achievement (2) Random Map Achievement (3) Cross-Map Achievement (4)
Beta Map Achievement (1) Ratings Achievement (3) Tournament Achievement (15) General Achievement (11) Challenge Achievement (1)

Re: Reinterpret ratings

Postby macbone on Tue Dec 04, 2012 6:45 am

I see where you're coming from, Sundance, but your rating is great, a 4.8. I'm not sure why you're upset with the current system.

Yeah, I know it doesn't match the information under Instructions - Ratings, but when I see a player with a 3.5, I have a pretty good idea what to expect. The same with a 4.7.

And the tags are useful,too. If I see a bunch of Good Teammate, Good Strategy, Friendly, etc., I want to play with that person. If I see Vindictive, Complainer, Bad Strategy, I'm going to be cautious (but not necessarily rule out playing with that person - maybe (= ). Just looking at your first page of ratings, you're definitely the type of player I'd enjoy playing in a game, and just scanning the games we've played together, I can see I'm right.

It's like the dice, man - it all evens out in the end.
User avatar
Major macbone
SoC Training Instructor
SoC Training Instructor
 
Posts: 5273
Joined: Wed Jun 03, 2009 7:12 pm
Location: Looking out over Victoria Harbor, thinking about Middle Earth and how good my wife looks
Medals: 75
Standard Achievement (3) Doubles Achievement (3) Triples Achievement (3) Quadruples Achievement (3) Terminator Achievement (3)
Assassin Achievement (1) Manual Troops Achievement (1) Freestyle Achievement (1) Nuclear Spoils Achievement (2) Fog of War Achievement (3)
Trench Warfare Achievement (2) Speed Achievement (1) Teammate Achievement (2) Random Map Achievement (3) Cross-Map Achievement (3)
Beta Map Achievement (1) Ratings Achievement (4) Tournament Achievement (8) General Achievement (8) Clan Achievement (9)
Training Achievement (2) Tournament Contribution (2) General Contribution (7)

Re: Reinterpret ratings

Postby Metsfanmax on Tue Dec 04, 2012 4:01 pm

sundance123 wrote:
Little Witt wrote:I support! I think the current rating system is all messed up, and is in need of dire changes.


You are right about that. Community doesn't give a toss though.


viewtopic.php?f=535&t=123707

Macbone wrote:Yeah, I know it doesn't match the information under Instructions - Ratings, but when I see a player with a 3.5, I have a pretty good idea what to expect. The same with a 4.7.

And the tags are useful,too. If I see a bunch of Good Teammate, Good Strategy, Friendly, etc., I want to play with that person. If I see Vindictive, Complainer, Bad Strategy, I'm going to be cautious (but not necessarily rule out playing with that person - maybe (= ). Just looking at your first page of ratings, you're definitely the type of player I'd enjoy playing in a game, and just scanning the games we've played together, I can see I'm right.


The problem is that this is essentially a binary system. If you have 4.7 or better, most people think of you as a sportsmanlike player, whereas anything less and most people consider you rude. There is no natural spread that helps you gain better insight into the specifics of how a player is.
User avatar
Lieutenant Metsfanmax
Head Thinker
Head Thinker
 
Posts: 3707
Joined: Wed Apr 11, 2007 11:01 pm
Location: NY
Medals: 40
Standard Achievement (3) Doubles Achievement (2) Triples Achievement (1) Quadruples Achievement (1) Terminator Achievement (1)
Assassin Achievement (1) Manual Troops Achievement (2) Freestyle Achievement (1) Nuclear Spoils Achievement (1) Fog of War Achievement (3)
Trench Warfare Achievement (1) Speed Achievement (3) Teammate Achievement (1) Random Map Achievement (1) Cross-Map Achievement (1)
Battle Royale Achievement (1) Ratings Achievement (2) Tournament Achievement (1) General Achievement (6) Clan Achievement (2)
General Contribution (5)

Re: Reinterpret ratings

Postby sundance123 on Wed Dec 05, 2012 4:45 am



That is just redressing the same problem. I am proposing a fundamental shift in the meaning behind ratings not how they are scored. One which I believe reflects how the majority of users actually rate each other. I dont believe there is a natural spread that could help you decide to play someone or not. This is a minor tweak to make a broken system work.

I am not particularly upset at the current system I do get annoyed at the idea that someone who leaves an all 3 rating can claim they are not giving a bad rating and that after 1 game a newb can rate someone as average. Whats with that?
User avatar
Major sundance123
 
Posts: 495
Joined: Fri Oct 30, 2009 6:29 pm
Medals: 69
Standard Achievement (4) Doubles Achievement (2) Triples Achievement (1) Quadruples Achievement (2) Terminator Achievement (2)
Assassin Achievement (2) Manual Troops Achievement (2) Freestyle Achievement (3) Nuclear Spoils Achievement (3) Fog of War Achievement (4)
Trench Warfare Achievement (2) Speed Achievement (2) Teammate Achievement (2) Random Map Achievement (3) Cross-Map Achievement (4)
Beta Map Achievement (1) Ratings Achievement (3) Tournament Achievement (15) General Achievement (11) Challenge Achievement (1)

Re: Reinterpret ratings

Postby Metsfanmax on Wed Dec 05, 2012 9:30 am

The whole purpose of a rating scale is so that the numerical average of the ratings says something meaningful about the player. Making the number of stars equivalent to some "tag" describing the player defeats the purpose of such a scale. That is why I strongly prefer the suggestion I linked; it retains the idea of the number of stars given corresponding to the quality of the player in the game you played, and not necessarily how you feel about them as a person.
User avatar
Lieutenant Metsfanmax
Head Thinker
Head Thinker
 
Posts: 3707
Joined: Wed Apr 11, 2007 11:01 pm
Location: NY
Medals: 40
Standard Achievement (3) Doubles Achievement (2) Triples Achievement (1) Quadruples Achievement (1) Terminator Achievement (1)
Assassin Achievement (1) Manual Troops Achievement (2) Freestyle Achievement (1) Nuclear Spoils Achievement (1) Fog of War Achievement (3)
Trench Warfare Achievement (1) Speed Achievement (3) Teammate Achievement (1) Random Map Achievement (1) Cross-Map Achievement (1)
Battle Royale Achievement (1) Ratings Achievement (2) Tournament Achievement (1) General Achievement (6) Clan Achievement (2)
General Contribution (5)

Re: Reinterpret ratings

Postby macbone on Wed Dec 05, 2012 12:02 pm

I'd definitely be against a automatic rating of 0 in that system you linked to, metsfan. I play too many games to individually rate players, but if that were implemented, I'd feel the need to rate every single player (no thanks).

-2 to +2 is the same thing that we have now, with the addition of an "average" rating when one doesn't rate a player.
User avatar
Major macbone
SoC Training Instructor
SoC Training Instructor
 
Posts: 5273
Joined: Wed Jun 03, 2009 7:12 pm
Location: Looking out over Victoria Harbor, thinking about Middle Earth and how good my wife looks
Medals: 75
Standard Achievement (3) Doubles Achievement (3) Triples Achievement (3) Quadruples Achievement (3) Terminator Achievement (3)
Assassin Achievement (1) Manual Troops Achievement (1) Freestyle Achievement (1) Nuclear Spoils Achievement (2) Fog of War Achievement (3)
Trench Warfare Achievement (2) Speed Achievement (1) Teammate Achievement (2) Random Map Achievement (3) Cross-Map Achievement (3)
Beta Map Achievement (1) Ratings Achievement (4) Tournament Achievement (8) General Achievement (8) Clan Achievement (9)
Training Achievement (2) Tournament Contribution (2) General Contribution (7)

Re: Reinterpret ratings

Postby Metsfanmax on Wed Dec 05, 2012 12:20 pm

macbone wrote:I'd definitely be against a automatic rating of 0 in that system you linked to, metsfan. I play too many games to individually rate players, but if that were implemented, I'd feel the need to rate every single player (no thanks).

-2 to +2 is the same thing that we have now, with the addition of an "average" rating when one doesn't rate a player.


Yes, but that average rating is of crucial importance. If you decided not to rate a player, it's because you think they're not remarkable (i.e. average). I don't think anyone should feel obligated to actively rate everyone under the new system, unless you have a particular desire to compliment/insult every single person you play.
User avatar
Lieutenant Metsfanmax
Head Thinker
Head Thinker
 
Posts: 3707
Joined: Wed Apr 11, 2007 11:01 pm
Location: NY
Medals: 40
Standard Achievement (3) Doubles Achievement (2) Triples Achievement (1) Quadruples Achievement (1) Terminator Achievement (1)
Assassin Achievement (1) Manual Troops Achievement (2) Freestyle Achievement (1) Nuclear Spoils Achievement (1) Fog of War Achievement (3)
Trench Warfare Achievement (1) Speed Achievement (3) Teammate Achievement (1) Random Map Achievement (1) Cross-Map Achievement (1)
Battle Royale Achievement (1) Ratings Achievement (2) Tournament Achievement (1) General Achievement (6) Clan Achievement (2)
General Contribution (5)

Re: Reinterpret ratings

Postby sundance123 on Fri Dec 07, 2012 9:15 am

Metsfanmax wrote:The whole purpose of a rating scale is so that the numerical average of the ratings says something meaningful about the player.

agreed


Metsfanmax wrote:Making the number of stars equivalent to some "tag" describing the player defeats the purpose of such a scale.


Not what I am suggesting. The opposite, in fact, for example 'average' is a tag with pretensions toward objectivity. A newb cannot accrately apply a label such as average. 'I will never play this guy again' is a subjective rating of the experience of playing against them & anybody can accurately rate their experience.

Metsfanmax wrote:That is why I strongly prefer the suggestion I linked; it retains the idea of the number of stars given corresponding to the quality of the player in the game you played


Well so does my suggestion - and in a better reasoned way. The quality of a player can best be defined as the average subjective experience of playing against them not by comparing each individual's experience to their idiosyncratic ideal player as encourage by the current system.

Perhaps you would discuss the merits of other suggestions in their respective threads - they have no relation to my suggestion.
User avatar
Major sundance123
 
Posts: 495
Joined: Fri Oct 30, 2009 6:29 pm
Medals: 69
Standard Achievement (4) Doubles Achievement (2) Triples Achievement (1) Quadruples Achievement (2) Terminator Achievement (2)
Assassin Achievement (2) Manual Troops Achievement (2) Freestyle Achievement (3) Nuclear Spoils Achievement (3) Fog of War Achievement (4)
Trench Warfare Achievement (2) Speed Achievement (2) Teammate Achievement (2) Random Map Achievement (3) Cross-Map Achievement (4)
Beta Map Achievement (1) Ratings Achievement (3) Tournament Achievement (15) General Achievement (11) Challenge Achievement (1)

Re: Reinterpret ratings

Postby Metsfanmax on Fri Dec 07, 2012 11:21 am

sundance123 wrote:Not what I am suggesting. The opposite, in fact, for example 'average' is a tag with pretensions toward objectivity. A newb cannot accrately apply a label such as average. 'I will never play this guy again' is a subjective rating of the experience of playing against them & anybody can accurately rate their experience.


Yes, but by doing this you also eliminate the purpose of the rating scale. If one star means "I will never play this person again," there is no useful information to a person reading the average rating. It says something about the interpersonal relationship between the rater and the rated but doesn't strive for objectivity, which is what you want in a rating system that averages ratings from different players. If your only defense is that the objective system is weak because newbs cannot accurately assess playing skill objectively, your case is not strong, because this will only last for a few games until the newbies pick up on the skills of a typical CC player. We could also fix that system by just preventing new players from awarding ratings for the first X games, instead of tossing out the whole system entirely.

Well so does my suggestion - and in a better reasoned way. The quality of a player can best be defined as the average subjective experience of playing against them not by comparing each individual's experience to their idiosyncratic ideal player as encourage by the current system.


How does one average "I will never play this person again?" and "I would play again with this person?" Would the result be "I would probably play again with this person?" That makes no sense. How does one average "needs to learn a thing or two" with "I would play again with this person? If you rate them a 4, are you willing to play with them again? If so, you need to revisit the tag for number 5.

The reason your suggestion makes no sense is that the scale is supposed to sample a continuum of values from "bad" to "good." This is the way any rating scale works. There is no obvious correlation between "foed" and "this person is stupid and I hate them." Is the change in playing skill from "foed" to "this person is stupid and I hate them" the same as the change in playing skill from "this person is stupid and I hate them" to "this person is stupid?" Of course not; there's no way to meaningfully average such a quantity, because there is no spread of values that is equally sampled by your scale.

Now, I'm sympathetic to the claim that since people don't use the rating system the way it's intended, that it's currently meaningless if it is intended to be interpreted as I am suggesting (as a fairly sampled average of evaluations of the player). But changing the rating scale to something meaningless doesn't actually solve that problem, because you are now averaging over some binary property (e.g. "I hated this game" or "I didn't hate this game") and you might as well discard an averaged rating scale at that point in favor of a tag system.

Perhaps you would discuss the merits of other suggestions in their respective threads - they have no relation to my suggestion.


This is not correct. As you can see, that suggestion is sitting in Submitted. Your suggestion is mutually exclusive with that suggestion. Therefore, if you want this suggestion to be installed preferentially over that system, you need to explain why yours is better.
User avatar
Lieutenant Metsfanmax
Head Thinker
Head Thinker
 
Posts: 3707
Joined: Wed Apr 11, 2007 11:01 pm
Location: NY
Medals: 40
Standard Achievement (3) Doubles Achievement (2) Triples Achievement (1) Quadruples Achievement (1) Terminator Achievement (1)
Assassin Achievement (1) Manual Troops Achievement (2) Freestyle Achievement (1) Nuclear Spoils Achievement (1) Fog of War Achievement (3)
Trench Warfare Achievement (1) Speed Achievement (3) Teammate Achievement (1) Random Map Achievement (1) Cross-Map Achievement (1)
Battle Royale Achievement (1) Ratings Achievement (2) Tournament Achievement (1) General Achievement (6) Clan Achievement (2)
General Contribution (5)

Re: Reinterpret ratings

Postby sundance123 on Fri Dec 07, 2012 5:55 pm

Metsfanmax wrote:There is no useful information to a person reading the average rating. It says something about the interpersonal relationship between the rater and the rated


A rating scale is subjective not objective - how the rating scale is used at the moment by raters in CC reflects "the interpersonal relationship between the rater and the rated" and this is not well served by the rating descriptions.

Any rating scale that ignores how users implement it is always going to miss the point whatever tricks you apply to obtain a nicer distribution.

What I am suggesting is that rating guidelines are changed to better reflect the intuitive logic applied by CCers to ratings.

Metsfanmax wrote: objectivity . . .what you want in a rating system that averages ratings from different players.


No you don't want that, when you say objectivity you are talking about ratings as interval level variables which they are not and will never be on CC.

The fact is the rating system serves people well at the moment for picking and choosing who they play against. It also generates C&A reports based on complaints of unfair ratings.

Metsfanmax wrote: If your only defense is . . . .


see above points


Metsfanmax wrote:How does one average . . . . blah blah blah


sundance123 wrote: *the exact wording to be worked out later.


Thanks for reading all of my original post. One of main points to be refined is how to describe ratings. And judging by the groundswell of support this suggestion has gotten ( Yes macbone I mean you :D ) we should have this point ironed out in a day or two!

Metsfanmax wrote: you are now averaging over some binary property (e.g. "I hated this game" or "I didn't hate this game")


Now this could be a better suggestion than mine and is actually is worth some discussion. Other CCers weigh in here please.


Metsfanmax wrote: you need to explain why yours is better.


Well so far only one person has trashed my idea and a lot more have trashed that one.
User avatar
Major sundance123
 
Posts: 495
Joined: Fri Oct 30, 2009 6:29 pm
Medals: 69
Standard Achievement (4) Doubles Achievement (2) Triples Achievement (1) Quadruples Achievement (2) Terminator Achievement (2)
Assassin Achievement (2) Manual Troops Achievement (2) Freestyle Achievement (3) Nuclear Spoils Achievement (3) Fog of War Achievement (4)
Trench Warfare Achievement (2) Speed Achievement (2) Teammate Achievement (2) Random Map Achievement (3) Cross-Map Achievement (4)
Beta Map Achievement (1) Ratings Achievement (3) Tournament Achievement (15) General Achievement (11) Challenge Achievement (1)

Re: Reinterpret ratings

Postby GeneralRisk on Fri Dec 07, 2012 6:18 pm

I think the current system is fine. 4.9 to 5.0 are players that seldom if ever play speed games and act nice in game chat. 4.3 to 4.8 are average players that are capable of both good and bad. 4.2 and lower are players that usually have issues and if you play them, then be prepared for either idiotic play and/or abusive game chat.
Major GeneralRisk
 
Posts: 762
Joined: Sun Jul 04, 2010 3:58 pm
Location: Neu-Schwabenland
Medals: 68
Standard Achievement (4) Doubles Achievement (4) Triples Achievement (2) Quadruples Achievement (2) Terminator Achievement (2)
Assassin Achievement (2) Manual Troops Achievement (3) Freestyle Achievement (1) Polymorphic Achievement (2) Nuclear Spoils Achievement (3)
Fog of War Achievement (4) Trench Warfare Achievement (3) Speed Achievement (4) Teammate Achievement (3) Random Map Achievement (2)
Cross-Map Achievement (3) Ratings Achievement (4) Tournament Achievement (11) Clan Achievement (9)

Re: Reinterpret ratings

Postby sundance123 on Fri Dec 07, 2012 7:08 pm

GeneralRisk wrote:I think the current system is fine. 4.9 to 5.0 are players that seldom if ever play speed games and act nice in game chat. 4.3 to 4.8 are average players that are capable of both good and bad. 4.2 and lower are players that usually have issues and if you play them, then be prepared for either idiotic play and/or abusive game chat.


I do agree with most of this - and this is a fairly useful guide to evaluating an opponent based on ratings - the problem I have is the instructions given for rating encourage people to give 3.0 rating to people who then feel they deserve higher because an average rating is not threes across the board it is two fives and a four!
User avatar
Major sundance123
 
Posts: 495
Joined: Fri Oct 30, 2009 6:29 pm
Medals: 69
Standard Achievement (4) Doubles Achievement (2) Triples Achievement (1) Quadruples Achievement (2) Terminator Achievement (2)
Assassin Achievement (2) Manual Troops Achievement (2) Freestyle Achievement (3) Nuclear Spoils Achievement (3) Fog of War Achievement (4)
Trench Warfare Achievement (2) Speed Achievement (2) Teammate Achievement (2) Random Map Achievement (3) Cross-Map Achievement (4)
Beta Map Achievement (1) Ratings Achievement (3) Tournament Achievement (15) General Achievement (11) Challenge Achievement (1)

Next

Return to Suggestions

Who is online

Users browsing this forum: No registered users and 1 guest

Login