Project Euler Forum

Hi PE,

I find that I am developing an unhealthy habit of not properly testing my code prior to submitting a solution. The reason being that the penalty for submitting a wrong answer is very low, I just have to wait a short while, which is typically shorter than the time I need to debug my code and perhaps write the unittests, which I "ought to" have written in the first place, if I were acting more professionally.

I think that if I were "rewarded" somehow by submitting a first correct answer, I would be more inclined to to develop code in a healthy process. I do not know if that is a goal in itself, but at least it is a quality, which is sought after in actual jobs where you do coding. The "reward" could be very simply like a statictics log in your profile, where you could see the percentage of successful submitted solutions. For me it would be a goal to keep a clean sheet, e.g., always submit the correct solution and spend a little more time checking out that everything is done right. Extending the idea, correctness could be a supplement to the speed metric which is measured for the Eulerians. I.e., the metric could be a based on speed and correctness. I think that would help stimulate more healthy problem solving habits.

Thoughts?

Slaunger

Perhaps we should penalise wrong entries? A few ideas...

(i) After entering more than three incorrect answers to a particular problem your account is automatically deleted.
(ii) Members enter a list of their family and friends' email addresses in their profile and each time the member guesses an answer incorrectly an email is automatically sent to the people on their list. The emails become increasingly insulting the more incorrect answers you enter.
(iii) We could manufacture USB devices which members purchase. Upon entering an incorrect answer the member - who is attached to the device - receives a brief electric shock. The intensity and length of the shock increases the more incorrect answers he/she enters.



On a more serious note... your suggestion is a great idea that would certainly motivate most members. However, and this may surprise you to hear this, there are some members who don't actually solve the problems. Instead they use "answer lists" found on the internet. So the problem is that the incentive of rewarding first time guesses for honest members would further encourage "cheaters", as not only would they be 100%ers - in terms of entering the correct answer to every problem - but they would appear to have solved every problem on their first attempt. And if the system was used as a metric then the cheaters would probably be ranked higher than the real solvers.

Hi Euler,

I understand the incentive to not motivate cheating, and I acknowledge that such a first-time right system could further stimulate cheating. Let me then propose a weaker proposal.

What if the stats for no of answers to get the right answer was not published anywhere but just kept internal. And once the correct answer for a problem was provided you are being given written appraisal for how well (or bad) you did it just for yourself. This would just be for yourself, and if you were cheating you would know it was not a "genuine" message.

Secondly, do the PE core members have any feeling how accessible cheats are for the newest problems? Like is it an issue for the last problems which go into the Eulerians ranking? If not it could still be part of the metric for those.

Of course one way to adress cheating would be to add a certain kind of diversity for the problem to solve, where the user id is used to adjust the specifics of the problem (e.g., there could be a handful of specific variations of the problem). I understand though that for some problems it is not easy to design the problem such that it can be solved for several special cases. Like I am working on problem 113 right now, which is to find the number of non-bouncy numbers under a gogool. It is hard to see how such a problem can be varied without spoiling the gogool detail. However, when it can be done it could be done. I acknowledge though that it would put further stress on the process of developing problems and the solution checking process because if the solution checking has a fault for one specific sub-solution but not for others it would be a terrible mess, which would make an unfair advantage for those members who had the fortune of being mapped to the correct subsolution.

Slaunger