Suggestion.. About ratings

Announcements, comments, ideas, feedback, and "How do I... ?" questions
Post Reply
nelapsi
Posts: 5
Joined: Tue Jan 15, 2008 12:45 pm
Contact:

Suggestion.. About ratings

Post by nelapsi »

At the moment rating of the is user is just a number of solved problems which is a bit unfair. For example, if user A solved Problem75 (which have been solved by about 540 users) and user B solved Problem1 (solved by more than 15000 users and it can be solved even without pen andd paper). Now these 2 users will have the same rating.
So I think it might be a good idea to create alternative rating (and alternative top1000) which will take in account difficulty of the solved tasks.
For example let
1) U - number of users who solved at least one task
2) ui - number of users who solved i-th task.
2) pi = U - ui
Then we can calculate user's rating like R =[sum]log(pi) (sum over all solved tasks).
Also we can calculate "month" rating (in this case U will be a number of users who solved at least one task during last month - i.e. "active" users)
PS or there might be several standard ratings systems or even some general engine wich will allow users to create own rating systems
WoLpH
Posts: 5
Joined: Thu Nov 01, 2007 3:38 pm

Re: Suggestion.. About ratings

Post by WoLpH »

The problem with something like this is mainly the period I guess.

The problems at the top of the list are considered to be easyer and tried more, does that make them easyer? For some yes, for others no.
The second point however is directly related to the amount of time the problem has been online and the people that have solved it, but that makes the rating a bit inacurate, the newest problems have nearly no users that have solved it, although that doesn't mean it's really hard.

So... when calculating the difficulty of a problem it would probably be a good idea to take the date it was placed into consideration aswell :)
For example, problem 171 has 99 people that solved it, while problem 176 only has already has 78 people that solved it, this probably indicates that 176 is considered to be easyer than 171.
User avatar
hk
Administrator
Posts: 12831
Joined: Sun Mar 26, 2006 10:34 am
Location: Haren, Netherlands

Re: Suggestion.. About ratings

Post by hk »

Until about a year ago we had a dynamic rating system for users ranks and problem difficulty.
About that time the number of solvers already became increasing faster and faster and the dynamic system began to influence server speed so with pain in our heart we disabled it.
Since then we have discussed several semi static rating systems but until now we did not find one that covered our situation well enough.
In the year that followed the speed of increase of the number of members has only been growing every month until about onr or two weeks ago the whole database began to bog and we had to introduce other restraints on the database use.
So I don't know, even when we can think up a valid rating system, we can afford to burden the server with the increased load.

Currently, to be on the lowest page of the top thousand one has had to solve 56 problems. I assume that by then the easiest problems will have been solved by that users.
Image
War ruins the life and health of untold numbers of innocent children.
nelapsi
Posts: 5
Joined: Tue Jan 15, 2008 12:45 pm
Contact:

Re: Suggestion.. About ratings

Post by nelapsi »

well, it might be a good idea to take in account task's "age"...
But just as I said there might be several rating systems and user can choose which of them is "better"...
Regarding "age" - as well we can take in account, for example, number of wrong answers for this task, or number of users, who entered answer (right or wrong), or smth else...
Anyway
Our 1st aim, I think, is to show what other rating systems are needed (in addition to the existing one)
Then, if we get administrations consent, we can discuss how to calculate rating
nelapsi
Posts: 5
Joined: Tue Jan 15, 2008 12:45 pm
Contact:

Re: Suggestion.. About ratings

Post by nelapsi »

about server loading..
I can see at least 2 ways:
1) calculate our ratings once per day (yep, this can hard enough too)
2) once per day download all required information to the separate database, which is used only for ratings

PS Existing rating is really unfair, because for me most of tasks don't seem to be interest but I want to be in the top1000 8-)
Last edited by nelapsi on Tue Jan 15, 2008 2:51 pm, edited 1 time in total.
User avatar
hk
Administrator
Posts: 12831
Joined: Sun Mar 26, 2006 10:34 am
Location: Haren, Netherlands

Re: Suggestion.. About ratings

Post by hk »

More than one rating system?
Did I not make it clear enough that we had severe problems with the serverload in the current situation?
Using more than one system would make that only worse.
Currently we are recovering from the fact that our provider disabled our entire database due to excess queries to it.
I'm afraid we will have to do with the current system until we are
1) sure that the load on the database stays manageable
2) we are sure that what we can think up one is better
3) what we can think up needs only updating once a week or once a month.
4) what we can think up does not increase the serverload in any way.
Image
War ruins the life and health of untold numbers of innocent children.
User avatar
hk
Administrator
Posts: 12831
Joined: Sun Mar 26, 2006 10:34 am
Location: Haren, Netherlands

Re: Suggestion.. About ratings

Post by hk »

nelapsi wrote:PS Existing rating is really, because for me most of tasks don't seem to be interest but I want to be in the top1000 8-)
Just solve at least 56 problems.
Image
War ruins the life and health of untold numbers of innocent children.
nelapsi
Posts: 5
Joined: Tue Jan 15, 2008 12:45 pm
Contact:

Re: Suggestion.. About ratings

Post by nelapsi »

hk wrote:Just solve at least 56 problems.
I'm too lazy to solve uninteresting tasks :)
nelapsi
Posts: 5
Joined: Tue Jan 15, 2008 12:45 pm
Contact:

Re: Suggestion.. About ratings

Post by nelapsi »

But with current number of active users even with "fair" rating system I probably won't have enough free time to solve enough interesting tasks to get to the top1000 :)
User avatar
euler
Administrator
Posts: 5094
Joined: Sun Mar 05, 2006 4:49 pm
Location: Cheshire, England
Contact:

Re: Suggestion.. About ratings

Post by euler »

I understand your concerns entirely, nelapsi, and one of our senior team members, hk, has already explained the dilemmas we face. Certainly as we add more and more problems to the database it will become increasingly difficult for new members to rise into the top 1000 scorers. We are constantly discussing better ways to score problems and rank members, but for the moment at least we need to ensure that the massive overhaul of the scripts have managed to tame the server load. Once we have established the the new scripts present a consistent and reasonable demand on the servers we will be in a better position to try out a new system.

However, in the meantime there is no harm in discussing ideas here so that when/if we establish the servers can cope with additional demand we will be better placed to implement a new system.
Image
impudens simia et macrologus profundus fabulae
Post Reply