Map Performance Stats
Posted: Fri Aug 15, 2008 6:15 pmThere are a number of threads dealing with the issue of map-specific performance. Bitzaholic has published stats on winning percentages, Daydream has listed top performers in terms of nett points won, to name but a couple. I have to say that although I feature at or near the top of some of these lists I have reservations about their value in assessing performance.
Winning percentage alone does not take strength of opposition into account.
Nett points won is biased in favour of players who have played more games. It is also biased by one's own overall rating. For example a 1000 rated player will have a far greater nett points gain than a 3000 rated player with the same performance against the same opposition on the map in question. I accept that rating is a statistically sound measure of overall performance, but when considering map-specific performance, one's general rating should not have influence. One needs to make the assumption that on average, opponents' general ratings are representative of their map-specific skill. But this is statistically justified with sufficient sample size. (Of 50 opponents, some may have general ratings in excess of their specific map skill, others may have general ratings which understate their specific map skill, but on average there is no expected bias.)
A further drawback of the above systems is they are based on one's entire playing history. It would be nice to have a measure of current skill, say over the last 50 games. The overall scoring system does meet this criterion in that recent results are far more influential than older results.
A solid measure of map performance should, in my opinion, take the following into consideration:
1) Ignore explicitly and implicitly the studied player's general points.
2) Consider strength of opposition.
3) Take only recent games into effect.
I propose 2 solutions. A is more statistically rigorous. B is more pragmatic.
Both address the following question: Given the last 50 results for this player on this map and given the opposition faced, if the player had played no other maps during this period, what rating would sustained, without nett gain or loss, by this performance?
Method A:
Let the effective map performance rating for player p on map m be R(0)
Let the ratings of opponents for each of the last 50 games g=1....50 be O(g)
If our player p had started with rating R, is rating after game 1 would be
R1 = R(0)+20.O(1)/R(0) for a win or
R1 = R(0)-20.R(0)/0(1) for a loss. (Modify these appropriately for multi-player and team games - I wont bore you with all the detail here - but can post later if required)
similarly R2, R3.....R50 can be recursively calculated in terms of R0
Set R50 = R0 and solve. The solution (R0) is the effective map rating.
Method A is rather laborious, so here is Method B which is a fair approximation of Method A:
Method B:
Let R be the player's effective map performance (to be calculated.)
Let w be the proportion of wins over the last 50 games.
Let O be the average opponents' rating over these same 50 games.
Then to maintain equilibrium of rating we require (approximately):
R + w.20.O/R - (1-w).20.R/0 = R
i.e. R = O.sqrt[w/(1-w)]
R is the effective map performance.
This is a readily implementable formula. It is statistically unbiased except where w = 1 or very close to 1(100% record.) I suggest a default in this regard of R = 5000 if w = 1.
I haven't the time to collect the data to calculate honours lists on this basis. But if someone feels up to writing a grease-monkey script, I believe this will be of some considerable value.
What thoughts?
Harry
Winning percentage alone does not take strength of opposition into account.
Nett points won is biased in favour of players who have played more games. It is also biased by one's own overall rating. For example a 1000 rated player will have a far greater nett points gain than a 3000 rated player with the same performance against the same opposition on the map in question. I accept that rating is a statistically sound measure of overall performance, but when considering map-specific performance, one's general rating should not have influence. One needs to make the assumption that on average, opponents' general ratings are representative of their map-specific skill. But this is statistically justified with sufficient sample size. (Of 50 opponents, some may have general ratings in excess of their specific map skill, others may have general ratings which understate their specific map skill, but on average there is no expected bias.)
A further drawback of the above systems is they are based on one's entire playing history. It would be nice to have a measure of current skill, say over the last 50 games. The overall scoring system does meet this criterion in that recent results are far more influential than older results.
A solid measure of map performance should, in my opinion, take the following into consideration:
1) Ignore explicitly and implicitly the studied player's general points.
2) Consider strength of opposition.
3) Take only recent games into effect.
I propose 2 solutions. A is more statistically rigorous. B is more pragmatic.
Both address the following question: Given the last 50 results for this player on this map and given the opposition faced, if the player had played no other maps during this period, what rating would sustained, without nett gain or loss, by this performance?
Method A:
Let the effective map performance rating for player p on map m be R(0)
Let the ratings of opponents for each of the last 50 games g=1....50 be O(g)
If our player p had started with rating R, is rating after game 1 would be
R1 = R(0)+20.O(1)/R(0) for a win or
R1 = R(0)-20.R(0)/0(1) for a loss. (Modify these appropriately for multi-player and team games - I wont bore you with all the detail here - but can post later if required)
similarly R2, R3.....R50 can be recursively calculated in terms of R0
Set R50 = R0 and solve. The solution (R0) is the effective map rating.
Method A is rather laborious, so here is Method B which is a fair approximation of Method A:
Method B:
Let R be the player's effective map performance (to be calculated.)
Let w be the proportion of wins over the last 50 games.
Let O be the average opponents' rating over these same 50 games.
Then to maintain equilibrium of rating we require (approximately):
R + w.20.O/R - (1-w).20.R/0 = R
i.e. R = O.sqrt[w/(1-w)]
R is the effective map performance.
This is a readily implementable formula. It is statistically unbiased except where w = 1 or very close to 1(100% record.) I suggest a default in this regard of R = 5000 if w = 1.
I haven't the time to collect the data to calculate honours lists on this basis. But if someone feels up to writing a grease-monkey script, I believe this will be of some considerable value.
What thoughts?
Harry
