Idea for Measuring Problem Difficulty

[Waldovski]

Hi, new member to the forum (but an old one to PE).

To be honest, I'm not a fan of the way difficulty is measured right now on PE. I would suggest using something like [frac]Number of users who solved the problem,number of days the problem has been on[/frac] and then normalising it against the hardest problem. Yes it will be skewed the first week or so thanks to all the geniuses here (intended as a compliment not a dig), but in the long run I think that's a better indicator of difficulty. You can even not give new problems a rating until a couple of weeks later.

Thanks for reading!

[harryh]

To be honest, I'm not a fan of the way difficulty is measured right now on PE.

As far as I can tell, we are NOT measuring difficulty anywhere; that has been abolished years ago!
Where did you see anything even remotely resembling a "problem rating"?

[Waldovski]

To be honest, I'm not a fan of the way difficulty is measured right now on PE.

As far as I can tell, we are NOT measuring difficulty anywhere; that has been abolished years ago!
Where did you see anything even remotely resembling a "problem rating"?

It's done implicitly when you "Sort problems in ... descending difficulty, or ascending difficulty".

[harryh]

Ah yes, you are right! Those expressions are a remnant of an old era.

"descending difficulty" should have been replaced by "ascending number of solutions" and
"ascending difficulty" should have been replaced by "descending number of solutions",
since that's exactly what those selections do.
The shorter expressions currently in use, simply reflect an item posted in the "About page":

How do I know where should I start?
That depends on your background. In the Problems table you will be able to see how many people have solved each problem. As a general rule of thumb the more people that have solved it, the easier it is.

As to modifying the sorting rules and using some "difficulty criterion" like the one you have suggested, I'm afraid that there are so many other factors affecting the number of solutions a problem gets, that it would be almost impossible to reach concensus...

[Waldovski]

While I disagree that there is any need for this measure to be any more complicated than the average number of solutions per day, I thank you for the quick replies.

[otac0n]

I would also like to see a more elaborate difficulty ranking.

We could hold an STV election if a consensus couldn't be reached.

Right now the ballot is:

1. [_] Solved (lower is harder)
2. [_] Solved / Age (lower is harder)

[Lord_Farin]

3 [_] Solved / Min(Age, 1000) (lower is harder)
4 [_] Solved / ((Average level of solvers) * Min(Age, 1000)) (lower is harder)

The Min is to prevent old problems like problem 1 to get harder because there are not enough users left that haven't solved it. Of course, the limit could be adapted.
I would like to see the average level of solvers in there because in this way we take into account some kind of measure on the average skill of the solver.

[otac0n]

Actually, I think that I have a better solution.

If we consider every solution that each person submits to be a vote for "I think that this question is easy," we can use each person's track record to simulate an "election."

This is, of course, what is happening now, if each person gets as may votes as they have answers:


However, if we were to divide the person's votes among their answers such that their total vote is equal to 1, we can build a summation of everybody's approximate skill level.

This has the effect of diluting people's contribution to Problem 001, if they have also solved a lot of other problems.

Here are some preliminary results (based on the first 100 people in the Level 1, 2, and 3 score tables):

http://files.otac0n.com/Results.txt

On top of that, we could decay votes based on the length of time since a user last answered a question. This would allow the newer questions to be more accurately ranked.

What do yall say?

[Lord_Farin]

Interesting concept, and as far as I can see, there are no major discrepancies in your txt file. However, I disagree on your last statement about decaying votes. I think that doing that results only in reducing the influence of inactive/unproductive users. This also is a bit tricky with the solution submission limits that are around. So I think we could best examine if the txt you gave is any good, and if it is, we could make a test run.

However, one could do the following:

Compute the ranking as you did, and then multiply it by (1+Age/1000) where again, 1000 can be adapted to suit the needs.

Let me not forget to compliment you with your work

[otac0n]

...
I think that doing that results only in reducing the influence of inactive/unproductive users.
...

I think that it would do both, and I think that that is a Good Thing.

Users who are not active have not had a chance to "vote" on the newest problems.