How should competition scores be calculated?
For those who are unaware, we run weekly skilling competitions in which participants are awarded a certain number of points based on a formula. These points are summed at the end of 12 weeks and those with the most points participate in a "playoff" to crown a champion.
However, our current formula has some major issues:
- Bonus experience allows some users to gain disproportionate amounts of xp
- Competitors are rewarded when less than 10 players actively participate
- One subscore is normalized based on xp rate, but I'm not always the best authority on a skill's xp rate
With season 3 of our skill competitions beginning on July 1st, I think this is a good time to start having a serious discussion about the future of our competitions.
Primarily, I want to create a formula that is:
- Reasonable - Players should not be able to win the entire season because they excelled one week
- Fair - Players should be rewarded for what they do, not what other players do not do and each competition should be equally valuable
- Maintainable - I do not actively play anymore, so I shouldn't have to stay up-to-date with the game in order to keep the competitions fair
Regarding the last point, if we decide that normalizing by xp rate is completely necessary, I'll gladly pass on competition creation duties to someone who is more active than I am.
It's important to remember, though, that the only data we have available is how much xp each player gained during the competition. We have know way of knowing how long a player has played. We don't know if they used bonus xp or not. We don't know if they purchased keys (well, it'd be easy enough to tell, but that process would be separate from this process).
This is the formula I'm currently considering:
It looks a lot more complex than it really is. All it does is take the natural logarithm of each user's xp, divide it by the natural logarithm of the xp gained by everyone in the top 10, subtract the floor of the lowest ratio of the natural logarithms, and multiplies by 50.
The first step (finding the ratio of the natural logarithms) converts what is effectively a parabolic distribution into a more linear one. The next step (subtracting the floor of the lowest ratio of the natural logarithms) normalizes the data to the rest of the field. For example, a set of data ranging from 7.2 to 6.4 would be converted to 1.2 to .4 and one that ranges from 4.7 to 4.1 would be converted to .7 to .1, which does a good job of creating parity between competitions, while still enforcing individual performance. Finally, the score is multiplied by a coefficient to create a score that is similar to what the score would have been under the old system.
Here is a spreadsheet that shows the change in scores: https://docs.google.com/spreadsheets/d/ ... mVexbYFXo/
As you can see, there is a lot more parity and competitions reward a smaller variation in total points.
There are a number of imperfections, such as if the maximum ratio of natural logarithms is relatively low, but I haven't found a competition yet that would create that issue. The formula is also hard to predict. It's basically a random number generator that takes in a player's xp as a seed.
So, what are your thoughts on the old formula? Does the new one offer a substantial improvement? What questions, comments, or issues do you have with the new one?
I would really like to have a formula that everyone is satisfied with before we head into season 3, so please leave your feedback if you have any.