[Site] Normalize Player Ratings (Average = 0)

This forum is a storehouse for all Finalized Suggestions to be periodically reviewed during Feature Updates.

Moderators: Suggestions Team, Global Moderators

Forum rules
Please read the Community Guidelines before posting.

Should we change the rating system, or leave it as it is?

Yes, change it.
76
52%
No, leave it.
70
48%
 
Total votes : 146

Re: Normalize Player Ratings (Average = 0) [Stickied ~ TFO]

Postby Metsfanmax on Thu Sep 23, 2010 8:56 am

jrh_cardinal wrote:1) Who rates -2 to 2? When giving a movie review you give it 1-5 stars, or go into a restaurant that has those "How did we do?" cards, the ratings they have on there are 1-something (usually 5, maybe 3 or something). You don't use negative numbers


Who rates 1 to 5? The fact that the average score is like 4.7 indicates that people do not understand how to properly rate on the current scale.

2) The last day of the old system:
Player x plays 100 multiplayer public games at a time, against random people, most of whom don't give a crap about CC or ratings. He gets rated by 25% of them, and has a rating of 4.8 (667 people have rated him)

Player y only plays 40 private team games at a time in his clan, Team 2000+++ usergroup, and clan wars all against people who are serious about CC, their score, and their rating. He gets rated by 50% of them, and has a rating of 4.8 (667 people have rated him)

6 Months after implementation of new system:
Player x has finished 1000 new games, in which he played 1333 new opponents (remember, public games against random people). His trends continued, so he's now been rated by 1000 people, with a rating of 1.8 on the new scale. He's also not been rated by 1000 people, so he actually has a rating of 0.9

Player y has finished 400 new games, in which he only played 66 new opponents (2 clan wars, only a few changes in his clan and usergroup). his trends continue, so he has now been rated by 700 people, with a rating of 1.8 on the new scale. He has not been rated by 33 people, so he now has a rating of 1.75ish

Are these two players deserving of such disparity in their ratings?


If someone is being rated a lot highly, it probably means that this player is generally seen as exceptional (it's possible that he simply played a lot of people who rate regularly, but this is rather unlikely). The fact that Player x is not rated as frequently means that more players generally see Player x as average. So yes, these two players deserve this disparity in their ratings. That is the entire point of the system: to make it clear who is simply an average player, and who is seen as an exceptional player.
User avatar
Lieutenant Metsfanmax
Head Thinker
Head Thinker
 
Posts: 3701
Joined: Wed Apr 11, 2007 11:01 pm
Location: NY
Medals: 40
Standard Achievement (3) Doubles Achievement (2) Triples Achievement (1) Quadruples Achievement (1) Terminator Achievement (1)
Assassin Achievement (1) Manual Troops Achievement (2) Freestyle Achievement (1) Nuclear Spoils Achievement (1) Fog of War Achievement (3)
Trench Warfare Achievement (1) Speed Achievement (3) Teammate Achievement (1) Random Map Achievement (1) Cross-Map Achievement (1)
Battle Royale Achievement (1) Ratings Achievement (2) Tournament Achievement (1) General Achievement (6) Clan Achievement (2)
General Contribution (5)

Re: Normalize Player Ratings (Average = 0) [Stickied ~ TFO]

Postby jrh_cardinal on Thu Sep 23, 2010 2:35 pm

Metsfanmax wrote:Who rates 1 to 5? The fact that the average score is like 4.7 indicates that people do not understand how to properly rate on the current scale.

You're avoiding the point. No, people do not rate 3 as an average rating on this site. Who cares? Having them rate -2 to 2 wouldn't change that, and it's stupid because when you're rating/reviewing someone with stars, you don't go -2 to 2, you go 1 to 5. Can you imagine a newspaper saying "We gave this movie -1 stars"? How do you give -1 of something anyway, no, you give out positive numbers.

Metsfanmax wrote:If someone is being rated a lot highly, it probably means that this player is generally seen as exceptional (it's possible that he simply played a lot of people who rate regularly, but this is rather unlikely). The fact that Player x is not rated as frequently means that more players generally see Player x as average. So yes, these two players deserve this disparity in their ratings. That is the entire point of the system: to make it clear who is simply an average player, and who is seen as an exceptional player.

No, it means he plays people who give a damn about CC, his rating is the same. The same percentage of people rate him low as the other guy. They both have a 4.8 (or 1.8 ) with just ratings, so the same percentage of people rate them low as rate them high. If you still disagree, I'll change it.

Player X stays the same, keeps his 0.9 rating.

Player Y only gets rated by 1/4 of the people he plays (same percent as player x). He played 68 new people, so in total he's been rated by 684 people, and not rated by 51. He now has a rating of 1.675.


So the only difference between the two of them is the number of new opponents. Should Player X be punished for playing against more people?
Image
User avatar
Sergeant 1st Class jrh_cardinal
 
Posts: 2715
Joined: Sat May 16, 2009 7:15 pm
Medals: 74
Standard Achievement (3) Doubles Achievement (2) Triples Achievement (2) Quadruples Achievement (3) Terminator Achievement (1)
Assassin Achievement (2) Manual Troops Achievement (2) Freestyle Achievement (2) Nuclear Spoils Achievement (2) Fog of War Achievement (3)
Speed Achievement (2) Random Map Achievement (1) Cross-Map Achievement (3) Ratings Achievement (2) Tournament Achievement (7)
General Achievement (3) Clan Achievement (2) Tournament Contribution (30) General Contribution (2)

Re: Normalize Player Ratings (Average = 0) [Stickied ~ TFO]

Postby Victor Sullivan on Thu Sep 23, 2010 3:46 pm

jrh_cardinal wrote:
Metsfanmax wrote:Who rates 1 to 5? The fact that the average score is like 4.7 indicates that people do not understand how to properly rate on the current scale.

You're avoiding the point. No, people do not rate 3 as an average rating on this site. Who cares? Having them rate -2 to 2 wouldn't change that, and it's stupid because when you're rating/reviewing someone with stars, you don't go -2 to 2, you go 1 to 5. Can you imagine a newspaper saying "We gave this movie -1 stars"? How do you give -1 of something anyway, no, you give out positive numbers.

I don't see how this justifies your position in this argument. It's just a f-ing number. Besides, I think the -2 to 2 is better, cuz then negative means you're below average and positive means you're above average. My point is, you can't use this argument for why not to implement this. It's about the rating mechanic, not what numbers to use.
jrh_cardinal wrote:
Metsfanmax wrote:If someone is being rated a lot highly, it probably means that this player is generally seen as exceptional (it's possible that he simply played a lot of people who rate regularly, but this is rather unlikely). The fact that Player x is not rated as frequently means that more players generally see Player x as average. So yes, these two players deserve this disparity in their ratings. That is the entire point of the system: to make it clear who is simply an average player, and who is seen as an exceptional player.

No, it means he plays people who give a damn about CC, his rating is the same. The same percentage of people rate him low as the other guy. They both have a 4.8 (or 1.8 ) with just ratings, so the same percentage of people rate them low as rate them high. If you still disagree, I'll change it.

Player X stays the same, keeps his 0.9 rating.

Player Y only gets rated by 1/4 of the people he plays (same percent as player x). He played 68 new people, so in total he's been rated by 684 people, and not rated by 51. He now has a rating of 1.675.


So the only difference between the two of them is the number of new opponents. Should Player X be punished for playing against more people?

Your question is bad, as both yes and no would support your argument, the way it's asked. The correct question to use for this situation is, "Is it fair that Player X has a lower overall rating than Player Y?" And the answer is yes.

-Sully for -222
User avatar
Corporal Victor Sullivan
 
Posts: 7197
Joined: Mon Feb 08, 2010 8:17 pm
Location: Columbus, OH
Medals: 45
Standard Achievement (2) Doubles Achievement (1) Triples Achievement (1) Quadruples Achievement (3) Terminator Achievement (1)
Assassin Achievement (1) Manual Troops Achievement (3) Freestyle Achievement (3) Nuclear Spoils Achievement (1) Fog of War Achievement (2)
Speed Achievement (2) Teammate Achievement (2) Random Map Achievement (1) Cross-Map Achievement (1) Ratings Achievement (1)
General Achievement (9) Map Contribution (4) Tournament Contribution (1) General Contribution (6)

Re: Normalize Player Ratings (Average = 0) [Stickied ~ TFO]

Postby Metsfanmax on Thu Sep 23, 2010 4:39 pm

jrh_cardinal wrote:You're avoiding the point. No, people do not rate 3 as an average rating on this site. Who cares? Having them rate -2 to 2 wouldn't change that, and it's stupid because when you're rating/reviewing someone with stars, you don't go -2 to 2, you go 1 to 5. Can you imagine a newspaper saying "We gave this movie -1 stars"? How do you give -1 of something anyway, no, you give out positive numbers.


Are you intentionally being obtuse? This is fairly easy to figure out. If you don't like the idea of giving someone a -1, then just give them a rating from 1 to 5 and subtract 3.

The change to -2 to 2 is mostly cosmetic; the zero default ratings are the real issue.

No, it means he plays people who give a damn about CC, his rating is the same. The same percentage of people rate him low as the other guy. They both have a 4.8 (or 1.8 ) with just ratings, so the same percentage of people rate them low as rate them high. If you still disagree, I'll change it.

Player X stays the same, keeps his 0.9 rating.

Player Y only gets rated by 1/4 of the people he plays (same percent as player x). He played 68 new people, so in total he's been rated by 684 people, and not rated by 51. He now has a rating of 1.675.


So the only difference between the two of them is the number of new opponents. Should Player X be punished for playing against more people?


It's not a question of punishment. If a given player is not rated very often, it means he or she is not exceptional, and so it makes sense that his or her rating falls near the average.

Also, I've pointed out that the only way this system will work is if we erase all previous ratings before transitioning to this new system.
User avatar
Lieutenant Metsfanmax
Head Thinker
Head Thinker
 
Posts: 3701
Joined: Wed Apr 11, 2007 11:01 pm
Location: NY
Medals: 40
Standard Achievement (3) Doubles Achievement (2) Triples Achievement (1) Quadruples Achievement (1) Terminator Achievement (1)
Assassin Achievement (1) Manual Troops Achievement (2) Freestyle Achievement (1) Nuclear Spoils Achievement (1) Fog of War Achievement (3)
Trench Warfare Achievement (1) Speed Achievement (3) Teammate Achievement (1) Random Map Achievement (1) Cross-Map Achievement (1)
Battle Royale Achievement (1) Ratings Achievement (2) Tournament Achievement (1) General Achievement (6) Clan Achievement (2)
General Contribution (5)

Re: Normalize Player Ratings (Average = 0) [Stickied ~ TFO]

Postby TheForgivenOne on Thu Sep 23, 2010 5:48 pm

Metsfanmax wrote:It's not a question of punishment. If a given player is not rated very often, it means he or she is not exceptional, and so it makes sense that his or her rating falls near the average.


Or, possibly this player plays someone who has reached Gold in Ratings, and has stopped rating altogether, OR has just played players who don't rate. Period.
Image
User avatar
Sergeant 1st Class TheForgivenOne
 
Posts: 5154
Joined: Fri May 15, 2009 8:27 pm
Location: Lost somewhere in the snow. HELP ME
Medals: 82
Standard Achievement (4) Doubles Achievement (3) Triples Achievement (3) Quadruples Achievement (3) Terminator Achievement (3)
Assassin Achievement (3) Manual Troops Achievement (3) Freestyle Achievement (4) Nuclear Spoils Achievement (3) Fog of War Achievement (4)
Trench Warfare Achievement (2) Speed Achievement (3) Teammate Achievement (2) Random Map Achievement (2) Cross-Map Achievement (3)
Beta Map Achievement (1) Battle Royale Achievement (2) Ratings Achievement (3) General Achievement (14) Clan Achievement (6)
General Contribution (11)

Re: Normalize Player Ratings (Average = 0) [Stickied ~ TFO]

Postby jrh_cardinal on Thu Sep 23, 2010 7:49 pm

Metsfanmax wrote:
jrh_cardinal wrote:No, it means he plays people who give a damn about CC, his rating is the same. The same percentage of people rate him low as the other guy. They both have a 4.8 (or 1.8 ) with just ratings, so the same percentage of people rate them low as rate them high. If you still disagree, I'll change it.

Player X stays the same, keeps his 0.9 rating.

Player Y only gets rated by 1/4 of the people he plays (same percent as player x). He played 68 new people, so in total he's been rated by 684 people, and not rated by 51. He now has a rating of 1.675.


So the only difference between the two of them is the number of new opponents. Should Player X be punished for playing against more people?


It's not a question of punishment. If a given player is not rated very often, it means he or she is not exceptional, and so it makes sense that his or her rating falls near the average.

great, you totally just oversupported my point :) . Percentage-wise they get rated the same (in the changed scenario), but Player X is "rated more often" because he plays more new people, so by your account he should have a higher rating, clearly he doesn't

Metsfanmax wrote:Also, I've pointed out that the only way this system will work is if we erase all previous ratings before transitioning to this new system.
[/quote]
That would work better, but then you lose all your old ratings, and lets be honest here, most people don't rate because they want to. If you take away everyones medal, they're just going to get pissed, then eventually rate the next 500 people they play perfect scores, which ruins your system. If you don't take away people's medals, they're just going to not rate people that they already rated once (even if they're an amazing player), because they don't want to rate. Everyone's rating would still be about the same.
Erasing the ratings is better, but I still don't see how it improves the situation. Let's say it works exactly as you want it to, people start rating 0 as average, so the average player's rating is 0. Great. The 99th percentile will be at like 0.2. What's the difference between having the spread from 4.5-4.9, and having it -.2 to .2?
This suggestion isn't going to make people give out an even number of -2's, -1's, 0's, 1's, and 2's; at the very best it's going to make the people going for the ratings medal rate all 0's (average) rather than all 5's (high).
Image
User avatar
Sergeant 1st Class jrh_cardinal
 
Posts: 2715
Joined: Sat May 16, 2009 7:15 pm
Medals: 74
Standard Achievement (3) Doubles Achievement (2) Triples Achievement (2) Quadruples Achievement (3) Terminator Achievement (1)
Assassin Achievement (2) Manual Troops Achievement (2) Freestyle Achievement (2) Nuclear Spoils Achievement (2) Fog of War Achievement (3)
Speed Achievement (2) Random Map Achievement (1) Cross-Map Achievement (3) Ratings Achievement (2) Tournament Achievement (7)
General Achievement (3) Clan Achievement (2) Tournament Contribution (30) General Contribution (2)

Re: Normalize Player Ratings (Average = 0) [Stickied ~ TFO]

Postby Metsfanmax on Thu Sep 23, 2010 9:51 pm

TheForgivenOne wrote:
Metsfanmax wrote:It's not a question of punishment. If a given player is not rated very often, it means he or she is not exceptional, and so it makes sense that his or her rating falls near the average.


Or, possibly this player plays someone who has reached Gold in Ratings, and has stopped rating altogether, OR has just played players who don't rate. Period.


It's statistically unlikely that a given player will play more non-raters on average, than any other given player.

jrh_cardinal wrote:great, you totally just oversupported my point . Percentage-wise they get rated the same (in the changed scenario), but Player X is "rated more often" because he plays more new people, so by your account he should have a higher rating, clearly he doesn't


The frequency of rating is what's important. In order for Player X to be "rated more often," i.e. at a higher frequency, a greater percentage of the people who play him have to give ratings. If Player A plays 1000 people and is rated 5 by 800 of them, and not left ratings by the other 200, then he will have the same rating as Player B, who played 100 people and was rated 5 80 of them, and unrated by 20. So Player A doesn't gain an advantage or disadvantage simply because he plays more games. In your scenario, the players are not given ratings at the same rate, so it is not fair to draw the conclusion that you did.

jrh_cardinal wrote:That would work better, but then you lose all your old ratings, and lets be honest here, most people don't rate because they want to. If you take away everyones medal, they're just going to get pissed, then eventually rate the next 500 people they play perfect scores, which ruins your system. If you don't take away people's medals, they're just going to not rate people that they already rated once (even if they're an amazing player), because they don't want to rate. Everyone's rating would still be about the same.


I said earlier on in the thread that I wouldn't take away people's medals. The whole point of this suggestion is that if people don't change their rating habits, then you're going to see most ratings suddenly skew toward the center (as they should). If that doesn't motivate them to give ratings for the sake of it, then nothing will.

Erasing the ratings is better, but I still don't see how it improves the situation. Let's say it works exactly as you want it to, people start rating 0 as average, so the average player's rating is 0. Great. The 99th percentile will be at like 0.2. What's the difference between having the spread from 4.5-4.9, and having it -.2 to .2?
This suggestion isn't going to make people give out an even number of -2's, -1's, 0's, 1's, and 2's; at the very best it's going to make the people going for the ratings medal rate all 0's (average) rather than all 5's (high).


The hope of this suggestion is that this will force people to change their rating habits, because they don't want everyone to have a 0 rating. You're right - maybe I'm overestimating what this might do. But no one can deny that the current rating system is totally broken, and if there's a chance to fix the system, I don't see why we shouldn't take it.
User avatar
Lieutenant Metsfanmax
Head Thinker
Head Thinker
 
Posts: 3701
Joined: Wed Apr 11, 2007 11:01 pm
Location: NY
Medals: 40
Standard Achievement (3) Doubles Achievement (2) Triples Achievement (1) Quadruples Achievement (1) Terminator Achievement (1)
Assassin Achievement (1) Manual Troops Achievement (2) Freestyle Achievement (1) Nuclear Spoils Achievement (1) Fog of War Achievement (3)
Trench Warfare Achievement (1) Speed Achievement (3) Teammate Achievement (1) Random Map Achievement (1) Cross-Map Achievement (1)
Battle Royale Achievement (1) Ratings Achievement (2) Tournament Achievement (1) General Achievement (6) Clan Achievement (2)
General Contribution (5)

Re: Normalize Player Ratings (Average = 0) [Stickied ~ TFO]

Postby jrh_cardinal on Thu Sep 23, 2010 10:44 pm

Metsfanmax wrote:
TheForgivenOne wrote:
Metsfanmax wrote:It's not a question of punishment. If a given player is not rated very often, it means he or she is not exceptional, and so it makes sense that his or her rating falls near the average.


Or, possibly this player plays someone who has reached Gold in Ratings, and has stopped rating altogether, OR has just played players who don't rate. Period.


It's statistically unlikely that a given player will play more non-raters on average, than any other given player.

incorrect. you are assuming pure randomness, CC players are not random. This is why my first example worked. If you play against premium oficers in clans/usergroups, they are much more likely to rate because they simply care more about the game than some freemie private you meet in a random public game.

Metsfanmax wrote:
jrh_cardinal wrote:great, you totally just oversupported my point . Percentage-wise they get rated the same (in the changed scenario), but Player X is "rated more often" because he plays more new people, so by your account he should have a higher rating, clearly he doesn't


The frequency of rating is what's important. In order for Player X to be "rated more often," i.e. at a higher frequency, a greater percentage of the people who play him have to give ratings. If Player A plays 1000 people and is rated 5 by 800 of them, and not left ratings by the other 200, then he will have the same rating as Player B, who played 100 people and was rated 5 80 of them, and unrated by 20. So Player A doesn't gain an advantage or disadvantage simply because he plays more games. In your scenario, the players are not given ratings at the same rate, so it is not fair to draw the conclusion that you did.

You didn't read the change in my scenario. I changed it so that player y only got rated by 25% of opponents (because you said this, which as I already said isn't valid, but that's beside the point), the same percentage as player x. So thank you for pointing out that by your standards, my example is completely valid.

jrh_cardinal wrote:No, it means he plays people who give a damn about CC, his rating is the same. The same percentage of people rate him low as the other guy. They both have a 4.8 (or 1.8 ) with just ratings, so the same percentage of people rate them low as rate them high. If you still disagree, I'll change it.

Player X stays the same, keeps his 0.9 rating.

Player Y only gets rated by 1/4 of the people he plays (same percent as player x). He played 68 new people, so in total he's been rated by 684 people, and not rated by 51. He now has a rating of 1.675.


Metsfanmax wrote:
jrh_cardinal wrote:That would work better, but then you lose all your old ratings, and lets be honest here, most people don't rate because they want to. If you take away everyones medal, they're just going to get pissed, then eventually rate the next 500 people they play perfect scores, which ruins your system. If you don't take away people's medals, they're just going to not rate people that they already rated once (even if they're an amazing player), because they don't want to rate. Everyone's rating would still be about the same.


I said earlier on in the thread that I wouldn't take away people's medals. The whole point of this suggestion is that if people don't change their rating habits, then you're going to see most ratings suddenly skew toward the center (as they should). If that doesn't motivate them to give ratings for the sake of it, then nothing will.

Why would people be motivated to give ratings? They wouldn't be. And who cares if they're in the center or not, as I already said ratings ranging from -.2 to .2 tell you no more about a person than ratings ranging from 4.5 to 4.9.

Metsfanmax wrote:
jrh_cardinal wrote:Erasing the ratings is better, but I still don't see how it improves the situation. Let's say it works exactly as you want it to, people start rating 0 as average, so the average player's rating is 0. Great. The 99th percentile will be at like 0.2. What's the difference between having the spread from 4.5-4.9, and having it -.2 to .2?
This suggestion isn't going to make people give out an even number of -2's, -1's, 0's, 1's, and 2's; at the very best it's going to make the people going for the ratings medal rate all 0's (average) rather than all 5's (high).


The hope of this suggestion is that this will force people to change their rating habits, because they don't want everyone to have a 0 rating. You're right - maybe I'm overestimating what this might do. But no one can deny that the current rating system is totally broken, and if there's a chance to fix the system, I don't see why we shouldn't take it.

The rating system is not totally broken. No, you can't really pick out the exceptional people from the crowd, but you can pick out the bad people. Anyone below a 4.5 is almost always not someone you want to partner with. So, for instance, my clan can say you have to have a minimum rating of 4.6 to apply for membership, that weeds out the bad people. Then I can go in and look at tags, look at individual ratings and click on the games to get an idea of whether this is someone we want. In your system, you have to reset the ratings or else it won't work at all (as you said), then everyone's rating is centered around 0, and the highest ratings will be the people that have been rated the most, which has just as much to do as the type of games that you play as it does with your personality. That just makes the ratings sclae even worse as there is no point that comes close to separating the above average from the below average

and if people start behaving "like they should" according to you, then everyone's going to be bunched up within .1, maybe .2 of 0, with just a few outliers. So, you'll be able to pick out the outstanding and the really terrible, but won't have any distinction between the good and the bad, which is what is really necessary.
Image
User avatar
Sergeant 1st Class jrh_cardinal
 
Posts: 2715
Joined: Sat May 16, 2009 7:15 pm
Medals: 74
Standard Achievement (3) Doubles Achievement (2) Triples Achievement (2) Quadruples Achievement (3) Terminator Achievement (1)
Assassin Achievement (2) Manual Troops Achievement (2) Freestyle Achievement (2) Nuclear Spoils Achievement (2) Fog of War Achievement (3)
Speed Achievement (2) Random Map Achievement (1) Cross-Map Achievement (3) Ratings Achievement (2) Tournament Achievement (7)
General Achievement (3) Clan Achievement (2) Tournament Contribution (30) General Contribution (2)

Re: Normalize Player Ratings (Average = 0) [Stickied ~ TFO]

Postby Metsfanmax on Thu Sep 23, 2010 10:55 pm

jrh_cardinal wrote:incorrect. you are assuming pure randomness, CC players are not random. This is why my first example worked. If you play against premium oficers in clans/usergroups, they are much more likely to rate because they simply care more about the game than some freemie private you meet in a random public game.


Prove your assertion that officers in clans rate more than free private players. Saying that they "care about the game more" doesn't indicate that they're more likely to leave ratings, and isn't even necessarily true.


You didn't read the change in my scenario. I changed it so that player y only got rated by 25% of opponents (because you said this, which as I already said isn't valid, but that's beside the point), the same percentage as player x. So thank you for pointing out that by your standards, my example is completely valid.


Your scenario is invalid, because it was based on the assumption that old ratings would be kept in the new system, which they wouldn't be.


Why would people be motivated to give ratings? They wouldn't be. And who cares if they're in the center or not, as I already said ratings ranging from -.2 to .2 tell you no more about a person than ratings ranging from 4.5 to 4.9.


They would be motivated to give ratings because now, if people do nothing, they can easily have a 4.7 or 4.8. In the new system, there will be a collective drive (hopefully) to rate more so that people don't get a lower rating than they desire (since presumably everyone wants an objectively high rating - I think most people want to be 1.8 out of 2, not 0.3, even though 0.3 might be above average.

The rating system is not totally broken. No, you can't really pick out the exceptional people from the crowd, but you can pick out the bad people. Anyone below a 4.5 is almost always not someone you want to partner with. So, for instance, my clan can say you have to have a minimum rating of 4.6 to apply for membership, that weeds out the bad people. Then I can go in and look at tags, look at individual ratings and click on the games to get an idea of whether this is someone we want. In your system, you have to reset the ratings or else it won't work at all (as you said), then everyone's rating is centered around 0, and the highest ratings will be the people that have been rated the most, which has just as much to do as the type of games that you play as it does with your personality. That just makes the ratings sclae even worse as there is no point that comes close to separating the above average from the below average


The new system would have that too, plus you'd see the other end of the spectrum - the really good players.

and if people start behaving "like they should" according to you, then everyone's going to be bunched up within .1, maybe .2 of 0, with just a few outliers. So, you'll be able to pick out the outstanding and the really terrible, but won't have any distinction between the good and the bad, which is what is really necessary.


Sure, but it looks like based on your outlook we may never have that. I'm more hopeful though - if it's possible to achieve that, then this suggestion will bring us in the right direction.
User avatar
Lieutenant Metsfanmax
Head Thinker
Head Thinker
 
Posts: 3701
Joined: Wed Apr 11, 2007 11:01 pm
Location: NY
Medals: 40
Standard Achievement (3) Doubles Achievement (2) Triples Achievement (1) Quadruples Achievement (1) Terminator Achievement (1)
Assassin Achievement (1) Manual Troops Achievement (2) Freestyle Achievement (1) Nuclear Spoils Achievement (1) Fog of War Achievement (3)
Trench Warfare Achievement (1) Speed Achievement (3) Teammate Achievement (1) Random Map Achievement (1) Cross-Map Achievement (1)
Battle Royale Achievement (1) Ratings Achievement (2) Tournament Achievement (1) General Achievement (6) Clan Achievement (2)
General Contribution (5)

Re: Normalize Player Ratings (Average = 0) [Stickied ~ TFO]

Postby jrh_cardinal on Thu Sep 23, 2010 11:16 pm

Metsfanmax wrote:
jrh_cardinal wrote:incorrect. you are assuming pure randomness, CC players are not random. This is why my first example worked. If you play against premium oficers in clans/usergroups, they are much more likely to rate because they simply care more about the game than some freemie private you meet in a random public game.


Prove your assertion that officers in clans rate more than free private players. Saying that they "care about the game more" doesn't indicate that they're more likely to leave ratings, and isn't even necessarily true.

you know I can't reasonably do that, you prove that they aren't. Random people in public games are usually just here to have fun, play a couple games of r***. Players in clans/usergroups are obviously more active in at least some parts of the site because they participate in those parts of the site (minimally clans/usergroups). They are active in parts of the site other than just playing, therefore they are more likely to participate in a site function that does not involve playing (namely ratings).
and yes, I understand that doesn't totally prove anything, but i'm pretty sure logical people are okay accepting that more active people are more active. it's bs for you to try to stonewall me with crap like that.


Metsfanmax wrote:
jrh_cardinal wrote:You didn't read the change in my scenario. I changed it so that player y only got rated by 25% of opponents (because you said this, which as I already said isn't valid, but that's beside the point), the same percentage as player x. So thank you for pointing out that by your standards, my example is completely valid.


Your scenario is invalid, because it was based on the assumption that old ratings would be kept in the new system, which they wouldn't be.

Metsfanmax wrote:the players are not given ratings at the same rate, so it is not fair to draw the conclusion that you did.

your quotes are contradictory. The first one you said (bottom one), said that my scenario was invalid because they weren't given ratings at the same rate. I proved they were, so you changed your story.

but yes, if players are rated at the same rate, then my new scenario doesn't work.

Metsfanmax wrote:
jrh_cardinal wrote:Why would people be motivated to give ratings? They wouldn't be. And who cares if they're in the center or not, as I already said ratings ranging from -.2 to .2 tell you no more about a person than ratings ranging from 4.5 to 4.9.


They would be motivated to give ratings because now, if people do nothing, they can easily have a 4.7 or 4.8. In the new system, there will be a collective drive (hopefully) to rate more so that people don't get a lower rating than they desire (since presumably everyone wants an objectively high rating - I think most people want to be 1.8 out of 2, not 0.3, even though 0.3 might be above average.

Are you serious? Let me get this straight. You want to normalize player ratings, so that the average is 0. NOW people are supposed to be motivated to rate other people high so that the other people will return the favor so that everyone can keep their high ratings. YOU REALIZE THAT MEANS THE AVERAGE IS NO LONGER 0!!!! So are you saying that everything you said before this is total bs that means nothing?

Metsfanmax wrote:
jrh_cardinal wrote:The rating system is not totally broken. No, you can't really pick out the exceptional people from the crowd, but you can pick out the bad people. Anyone below a 4.5 is almost always not someone you want to partner with. So, for instance, my clan can say you have to have a minimum rating of 4.6 to apply for membership, that weeds out the bad people. Then I can go in and look at tags, look at individual ratings and click on the games to get an idea of whether this is someone we want. In your system, you have to reset the ratings or else it won't work at all (as you said), then everyone's rating is centered around 0, and the highest ratings will be the people that have been rated the most, which has just as much to do as the type of games that you play as it does with your personality. That just makes the ratings sclae even worse as there is no point that comes close to separating the above average from the below average


The new system would have that too, plus you'd see the other end of the spectrum - the really good players.

It's pointless to respond to this until you answer the above question

Metsfanmax wrote:
jrh_cardinal wrote:and if people start behaving "like they should" according to you, then everyone's going to be bunched up within .1, maybe .2 of 0, with just a few outliers. So, you'll be able to pick out the outstanding and the really terrible, but won't have any distinction between the good and the bad, which is what is really necessary.


Sure, but it looks like based on your outlook we may never have that. I'm more hopeful though - if it's possible to achieve that, then this suggestion will bring us in the right direction.

This statement directly contradicts your one a little earlier that says people's motivation to rate for no reason is that they will give others high ratings so that they can get themself a high rating.
Image
User avatar
Sergeant 1st Class jrh_cardinal
 
Posts: 2715
Joined: Sat May 16, 2009 7:15 pm
Medals: 74
Standard Achievement (3) Doubles Achievement (2) Triples Achievement (2) Quadruples Achievement (3) Terminator Achievement (1)
Assassin Achievement (2) Manual Troops Achievement (2) Freestyle Achievement (2) Nuclear Spoils Achievement (2) Fog of War Achievement (3)
Speed Achievement (2) Random Map Achievement (1) Cross-Map Achievement (3) Ratings Achievement (2) Tournament Achievement (7)
General Achievement (3) Clan Achievement (2) Tournament Contribution (30) General Contribution (2)

Re: Normalize Player Ratings (Average = 0) [Stickied ~ TFO]

Postby Metsfanmax on Thu Sep 23, 2010 11:43 pm

jrh_cardinal wrote:you know I can't reasonably do that, you prove that they aren't. Random people in public games are usually just here to have fun, play a couple games of r***. Players in clans/usergroups are obviously more active in at least some parts of the site because they participate in those parts of the site (minimally clans/usergroups). They are active in parts of the site other than just playing, therefore they are more likely to participate in a site function that does not involve playing (namely ratings).
and yes, I understand that doesn't totally prove anything, but i'm pretty sure logical people are okay accepting that more active people are more active. it's bs for you to try to stonewall me with crap like that.


No it's not. Your argument was that people who play in clans get rated more. But I'm not in a clan, and I still get rated by people who are ranked highly and are in clans, since those people don't just play clan games.

My real problem with your argument was the idea that someone in a clan is "more active" than someone who is not. That's not true - it simply means they have more "responsibilities," as it were. It says nothing about who they are as a person or how likely they are to rate, and says little about how much time they spend on CC.


your quotes are contradictory. The first one you said (bottom one), said that my scenario was invalid because they weren't given ratings at the same rate. I proved they were, so you changed your story.

but yes, if players are rated at the same rate, then my new scenario doesn't work.


They're not contradictory - they're just two different reasons why your scenario was invalid. One reason is that it assumed old ratings would be kept, and another reason is that it assumed different frequencies of being rated. The two arguments are not mutually exclusive, and they're certainly not mutually exclusive simply because I didn't say them at the same time.


Are you serious? Let me get this straight. You want to normalize player ratings, so that the average is 0. NOW people are supposed to be motivated to rate other people high so that the other people will return the favor so that everyone can keep their high ratings. YOU REALIZE THAT MEANS THE AVERAGE IS NO LONGER 0!!!! So are you saying that everything you said before this is total bs that means nothing?


The average will still be zero. What I'm saying is that people will be motivated to rate because they want people to rate them back. The reason why this won't make the average positive is that while most people would probably like to think that they deserve an above average rating, their opponents will not actually see it that way. People will be motivated to rate more because they want a higher rating, and some people won't get it, because the ideal distribution is a normal curve - but it doesn't matter why they rate more, as long as they do.

This statement directly contradicts your one a little earlier that says people's motivation to rate for no reason is that they will give others high ratings so that they can get themself a high rating.


See above. This will be their motivation to rate, but in reality they'll only get a high rating if they actually earn it (which isn't most people).
User avatar
Lieutenant Metsfanmax
Head Thinker
Head Thinker
 
Posts: 3701
Joined: Wed Apr 11, 2007 11:01 pm
Location: NY
Medals: 40
Standard Achievement (3) Doubles Achievement (2) Triples Achievement (1) Quadruples Achievement (1) Terminator Achievement (1)
Assassin Achievement (1) Manual Troops Achievement (2) Freestyle Achievement (1) Nuclear Spoils Achievement (1) Fog of War Achievement (3)
Trench Warfare Achievement (1) Speed Achievement (3) Teammate Achievement (1) Random Map Achievement (1) Cross-Map Achievement (1)
Battle Royale Achievement (1) Ratings Achievement (2) Tournament Achievement (1) General Achievement (6) Clan Achievement (2)
General Contribution (5)

Re: Normalize Player Ratings (Average = 0) [Stickied ~ TFO]

Postby jrh_cardinal on Fri Sep 24, 2010 12:44 am

Metsfanmax wrote:
jrh_cardinal wrote:you know I can't reasonably do that, you prove that they aren't. Random people in public games are usually just here to have fun, play a couple games of r***. Players in clans/usergroups are obviously more active in at least some parts of the site because they participate in those parts of the site (minimally clans/usergroups). They are active in parts of the site other than just playing, therefore they are more likely to participate in a site function that does not involve playing (namely ratings).
and yes, I understand that doesn't totally prove anything, but i'm pretty sure logical people are okay accepting that more active people are more active. it's bs for you to try to stonewall me with crap like that.


No it's not. Your argument was that people who play in clans get rated more. But I'm not in a clan, and I still get rated by people who are ranked highly and are in clans, since those people don't just play clan games.

My real problem with your argument was the idea that someone in a clan is "more active" than someone who is not. That's not true - it simply means they have more "responsibilities," as it were. It says nothing about who they are as a person or how likely they are to rate, and says little about how much time they spend on CC.

There are plenty of people who play few/no public games. My argument is that people who consistently play against people who are active in clans/usergroups/tournaments/foundry/any other part of cc besides just public games, get rated more. The most extreme example of this is clan people who play pretty much only clan wars, high-ranker usergroup practice games, stuff like that.

Newest active 8 man public game: Game 7683973. I did not pick any specific game that fit my interests, just the newest 8 man. The 8 players have a total combined post count of 6, none of them are in any clan or usergroup, and only one of them has anything more than a bronze rating medal.
Most recent started Team 2000+++ usergroup quads game (it's over now, but whatever): Game 7540522. I did not pick any specific game that fit my interests, just the newest quads game (only 8 man games in the team game usergroup). The lowest post count was in the 80s, highest in the 1000s. every one of them had a gold ratings medal (I'm pretty sure).

I think that's pretty good evidence that in general people that are active on forums and elsewhere on cc besides games care more about at least getting the ratings medal than random people playing random public games.

I didn't really mean that you have to be in a clan to be active and get rated a lot, it's just the best example because people in clans and usergroups are the ones that play the most private games against active opponents. It's the opponents that matter, since they're the ones doing the ratings.


Metsfanmax wrote:
jrh_cardinal wrote:Are you serious? Let me get this straight. You want to normalize player ratings, so that the average is 0. NOW people are supposed to be motivated to rate other people high so that the other people will return the favor so that everyone can keep their high ratings. YOU REALIZE THAT MEANS THE AVERAGE IS NO LONGER 0!!!! So are you saying that everything you said before this is total bs that means nothing?


The average will still be zero. What I'm saying is that people will be motivated to rate because they want people to rate them back. The reason why this won't make the average positive is that while most people would probably like to think that they deserve an above average rating, their opponents will not actually see it that way. People will be motivated to rate more because they want a higher rating, and some people won't get it, because the ideal distribution is a normal curve - but it doesn't matter why they rate more, as long as they do.

but the first people have to give high ratings right? I mean, if I go around giving out a bunch of low ratings I'm not convincing my opponents to rate me highly. So the only way for the chain to continue, and have the next person motivated to rate, is by giving out all or at least mostly good ratings.

Metsfanmax wrote:in reality they'll only get a high rating if they actually earn it (which isn't most people).

You magical chain of ratings will be broken wherever a low rating is left
Image
User avatar
Sergeant 1st Class jrh_cardinal
 
Posts: 2715
Joined: Sat May 16, 2009 7:15 pm
Medals: 74
Standard Achievement (3) Doubles Achievement (2) Triples Achievement (2) Quadruples Achievement (3) Terminator Achievement (1)
Assassin Achievement (2) Manual Troops Achievement (2) Freestyle Achievement (2) Nuclear Spoils Achievement (2) Fog of War Achievement (3)
Speed Achievement (2) Random Map Achievement (1) Cross-Map Achievement (3) Ratings Achievement (2) Tournament Achievement (7)
General Achievement (3) Clan Achievement (2) Tournament Contribution (30) General Contribution (2)

Re: Normalize Player Ratings (Average = 0) [Stickied ~ TFO]

Postby Metsfanmax on Fri Sep 24, 2010 1:00 am

jrh_cardinal wrote:Newest active 8 man public game: Game 7683973. I did not pick any specific game that fit my interests, just the newest 8 man. The 8 players have a total combined post count of 6, none of them are in any clan or usergroup, and only one of them has anything more than a bronze rating medal.
Most recent started Team 2000+++ usergroup quads game (it's over now, but whatever): Game 7540522. I did not pick any specific game that fit my interests, just the newest quads game (only 8 man games in the team game usergroup). The lowest post count was in the 80s, highest in the 1000s. every one of them had a gold ratings medal (I'm pretty sure).

I think that's pretty good evidence that in general people that are active on forums and elsewhere on cc besides games care more about at least getting the ratings medal than random people playing random public games.


No, it's anecdotal evidence. That never qualifies as "pretty good" evidence.

but the first people have to give high ratings right? I mean, if I go around giving out a bunch of low ratings I'm not convincing my opponents to rate me highly. So the only way for the chain to continue, and have the next person motivated to rate, is by giving out all or at least mostly good ratings.


Not at all. You're missing the forest for the trees. The point is that people don't want to just get an average rating, so they'll start giving ratings in the hopes that the system will change. If they end up with only an average rating afterward, then sure, they might give up on it, or they might keep working hard to improve their ratings. I think that the latter will be the case. It can't be any worse than the current system.

None of your remaining arguments are active reasons why we shouldn't do this - they're just reasons why we might potentially not change anything. I don't see why straight pessimism is useful here.
User avatar
Lieutenant Metsfanmax
Head Thinker
Head Thinker
 
Posts: 3701
Joined: Wed Apr 11, 2007 11:01 pm
Location: NY
Medals: 40
Standard Achievement (3) Doubles Achievement (2) Triples Achievement (1) Quadruples Achievement (1) Terminator Achievement (1)
Assassin Achievement (1) Manual Troops Achievement (2) Freestyle Achievement (1) Nuclear Spoils Achievement (1) Fog of War Achievement (3)
Trench Warfare Achievement (1) Speed Achievement (3) Teammate Achievement (1) Random Map Achievement (1) Cross-Map Achievement (1)
Battle Royale Achievement (1) Ratings Achievement (2) Tournament Achievement (1) General Achievement (6) Clan Achievement (2)
General Contribution (5)

Re: Normalize Player Ratings (Average = 0) [Stickied ~ TFO]

Postby MichelSableheart on Fri Sep 24, 2010 3:41 pm

@jrh: if you're playing consistently with players who care enough about the site to rate, wouldn't those players care enough about the site to rate accurately? Those players don't see the rating system as a friend's service, do they?

The way I see it, changing the rating system so that no rating = average rating, has two advantages. The first is that you are able to identify the exceptional players, in the same way you are currently able to identify the really poor players. This seems to me a positive thing, especially for clan recruitment and searching for new team partners.

The second change is a change in mentality. Currently, almost everyone is rated close to a 5. For someone who is new at rating, this creates a huge barrier for rating someone accurately (poor if you think they were poor, average if you think they were average, good if you think they were good). If everyone is rated closely to average, the incentive to rate someone all 5's is reduced, hopefully leading to more accurate ratings.
User avatar
Colonel MichelSableheart
 
Posts: 610
Joined: Sat Jun 27, 2009 5:10 pm
Medals: 21
Standard Achievement (2) Doubles Achievement (1) Triples Achievement (1) Quadruples Achievement (1) Manual Troops Achievement (1)
Fog of War Achievement (2) Speed Achievement (1) Teammate Achievement (1) Random Map Achievement (1) Cross-Map Achievement (1)
Ratings Achievement (3) Tournament Achievement (1) General Achievement (1) Clan Achievement (3) Tournament Contribution (1)

Re: Normalize Player Ratings (Average = 0) [Stickied ~ TFO]

Postby jrh_cardinal on Fri Sep 24, 2010 4:03 pm

Metsfanmax wrote:No, it's anecdotal evidence. That never qualifies as "pretty good" evidence.

It's statistical evidence. There is a very small statistical probablility that these two groups of players as a whole rate the same amount, given that in my example 1/8 public players rate frequently, and 8/8 usergroup players rate frequently

Metsfanmax wrote:Not at all. You're missing the forest for the trees. The point is that people don't want to just get an average rating, so they'll start giving ratings in the hopes that the system will change. If they end up with only an average rating afterward, then sure, they might give up on it, or they might keep working hard to improve their ratings. I think that the latter will be the case. It can't be any worse than the current system.

None of your remaining arguments are active reasons why we shouldn't do this - they're just reasons why we might potentially not change anything. I don't see why straight pessimism is useful here.

No, you are missing reality for utopia which isn't even utopia. People are not motivated to rate because people are rating them average scores. Hell, they probably don't even notice they're being rated if they're rating doesn't change, which it wouldn't if they're being rated average. You are way overestimating the CC public.

It's not straight pessimism, it's reality.

And don't say that it doesn't hurt to try, I already said that it does. There's an even smaller difference between the below average and above average, only the very extremes will show themselves


MichelSableheart wrote:@jrh: if you're playing consistently with players who care enough about the site to rate, wouldn't those players care enough about the site to rate accurately? Those players don't see the rating system as a friend's service, do they?

The simplest answer to this is they don't average 3. For me, I only rate if I have a really good or bad experience (or if I'm really bored so I decide to get closer to ther medal). In games like that, everyone is good, so it's rare to have a really bad experience

MichelSableheart wrote:The first is that you are able to identify the exceptional players, in the same way you are currently able to identify the really poor players. This seems to me a positive thing, especially for clan recruitment and searching for new team partners.

Yes, you are able to identify the truly exceptional players (if everything goes exactly according to your guys plan). I have accepted that as a fact in multiple posts before, to show that it still doesn't help. IF everyone starts rating as an average of 0 exactly like you guys want, 90+% of the people on the site are going to have a 0, because most of the 'ratings' you get will already be 0, so even if you are slightly above or below for real ratings, it will not show up in te final computation. With 5% of the population shown to be exceptional, that doesn't help clans. Right now, our minimum requirement is a 4.6. Approximately 2/3 of the site meets that opening requirement. There's not going to be anything CLOSE to a split at the 2/3 mark in your new system, everyone's going to have a 0, and that's assuming everything works perfectly according to you.
Image
User avatar
Sergeant 1st Class jrh_cardinal
 
Posts: 2715
Joined: Sat May 16, 2009 7:15 pm
Medals: 74
Standard Achievement (3) Doubles Achievement (2) Triples Achievement (2) Quadruples Achievement (3) Terminator Achievement (1)
Assassin Achievement (2) Manual Troops Achievement (2) Freestyle Achievement (2) Nuclear Spoils Achievement (2) Fog of War Achievement (3)
Speed Achievement (2) Random Map Achievement (1) Cross-Map Achievement (3) Ratings Achievement (2) Tournament Achievement (7)
General Achievement (3) Clan Achievement (2) Tournament Contribution (30) General Contribution (2)

PreviousNext

Return to Submitted Suggestions

Who is online

Users browsing this forum: No registered users and 1 guest

cron
Login