Cheaters at the top of the fastest solvers lists?

[universalset]

I'd assign about 90% chance to at least one of the top two placing accounts on the problems from last weekend (969 and 970) being cheaters (presumably using LLMs to assist, which are known to solve both of these problems).

- Both accounts solved 970 much faster than any other solvers. This includes some of our most talented users.
- The "Remilia" (display name for "HeavyCrab") account solved both problems in under 15 minutes, and within thirty seconds of each other.
- The "AlperTheKing" account solved 969 in under 7 minutes, which is, if not definitively cheating, still incredibly fast compared to even the fastest other solvers. Then it followed up with 970 less than 15 minutes later.
- Neither account had appeared on any Fastest Solvers list before September of this year.

I suppose it is possible that these are both legitimate solvers who just happened to hit problems that they immediately knew exactly how to solve. And that one of them decided to synchronize putting in answers to the two problems for some reason.

But I am extremely skeptical. Do we have any reason to believe that these are legitimate solve times?

(I was prompted to post this due to some skeptical discussion in the Discord server. It's not just me who's giving these a side-eye.)

[neverforget]

FYI there's highly incriminating evidence in the 971 forum historical posts if any admins really want to look into it. Sorry I forgot to grab receipts. It was so bad it was kind of entertaining, and the horrendous code is still there on display like some kind deranged tribute to vibe coding. 0% legit beyond all doubt.

[oblivyawn]

I've been doing PE since middle school - sometimes taking breaks for years and coming back. I've never much cared about leaderboard or wracking up solves. I only recently decided to try for the occasional leaderboard - after I saw one of the problems was blocked from leaderboard bc of leaks and I thought I should grab a few while the feature still exists

I agree that the timing is suspicious and the code I saw in the threads looked very sus. Idk what can be done - but what I wanted to say was - if the cheating people, whether by leaks of gen AI, are reading this, there are a lot of members who value that leaderboard position. I just don't understand why you'd deprive of them of that? You may think the PE developers philosophy is wrong, whatever, but why be unkind to people that value the leaderboards?

[echip]

If someone is using LLM to cheat, then at least they would have to put in considerable effort to generate correct code for LLM.

What concerns me more is that someone is exploiting their status as a problem author to gain undue advantage. That is, providing a question using another account and then answering it with their own original account.

There's a guy on the leaderboard whose highest ranking, excluding the top 10, is only 12th. His average ranking is around 30th~40th. But if he appears in the top 10, he's always 2nd. he had three 2nd-place recently. This looks like he knew the answer before the problem was published and waited there,then immediately submitted the correct answer after the first solver answered to become 2nd in the rankings.

Besides this guy, I'm sure there are others who have done this kind of thing.

[universalset]

If someone is using LLM to cheat, then at least they would have to put in considerable effort to generate correct code for LLM.

You have no idea what you are talking about; this is blatantly false. LLMs have been able to generate (and run) code to solve many PE problems for quite a while now, with no special effort required.

(And anyway, using an LLM to do the work on a problem is cheating, even if the user puts in some effort as well.)

What concerns me more is that someone is exploiting their status as a problem author to gain undue advantage. That is, providing a question using another account and then answering it with their own original account.

There's a guy on the leaderboard whose highest ranking, excluding the top 10, is only 12th. His average ranking is around 30th~40th. But if he appears in the top 10, he's always 2nd. he had three 2nd-place recently. This looks like he knew the answer before the problem was published and waited there,then immediately submitted the correct answer after the first solver answered to become 2nd in the rankings.

Besides this guy, I'm sure there are others who have done this kind of thing.

I'm sure that you are full of it. This sort of thing is technically possible, but your fixation on this avenue is bordering on a delusion. For one thing, it's way higher effort, way less likely to pay off, and has a way longer time delay to getting the reward (such as it is), than copy-pasting the problem statement into an LLM! If someone is willing to do such high-effort, risky cheating, why wouldn't they be willing to do the low-effort kind (which has some plausible deniability to boot)? None of your theory makes sense.

[echip]

I'm sure that you are full of it. This sort of thing is technically possible, but your fixation on this avenue is bordering on a delusion. For one thing, it's way higher effort, way less likely to pay off, and has a way longer time delay to getting the reward (such as it is), than copy-pasting the problem statement into an LLM! If someone is willing to do such high-effort, risky cheating, why wouldn't they be willing to do the low-effort kind (which has some plausible deniability to boot)? None of your theory makes sense.

If you don't believe me, fine. I wrote this for that person, not for you. I know you harbor deep hostility towards me.

That person knows I hit the nail on the head. Explaining it to you would be a waste of my breath. It's not a matter of higher effort or lower effort; it's that person is inherently interested in being a problem setter and believes he deserves a reward.

Seeing your own questions gloriously featured on ProjectEuler and earning you a high rank—that double satisfaction is something those who use LLM to cheat can never match. If you don't even understand this, we're truly not on the same level.

[hk]

Please read the latest news item.

[pjt33]

Please read the latest news item.

The item links to a URL /genai which displays identical content to /about , at least when not logged into projecteuler.net. I haven't checked it while logged in.

[hk]

Please read the latest news item.

The item links to a URL /genai which displays identical content to /about , at least when not logged into projecteuler.net. I haven't checked it while logged in.

Try it when logged in

[Lazar]

This thread was full of unempathy. "So as to be absolutely clear, we have..." from the Project Euler team was a good moment because evidently beforehand users had a misunderstanding. @euler below, this was it, in this thread people are casually throwing the word "cheating" around when the users in question plausibly had no clue about these discussions in the forum and Discord.

[euler]

I've added the generative AI tools policy to the list of about items that can be read without being signed in.

@Lazar: Thanks for the heads-up on the typo. I've fixed it now. I'm not quite sure about the rest of your post. Have I missed something else that is unclear?

[universalset]

Euler, hk, and the rest of the PE admin staff:

I really appreciate your adding this language about what uses of generative AI are acceptable (in particular making it clear that using it for learning about a topic is OK, but that having it tell you how to solve a problem or write the code for you is not). I'm probably more negative on AI than most (for various reasons), but this lines up with what I'd consider cheating or not on a Project Euler problem. I hope that having some official language on the site about this will help.

I also understand that policing for poor behavior (cheating, gumming up forums with AI code, etc.) is a difficult and thankless job, especially since there's rarely a true smoking gun.

When I wrote the original post I was very frustrated, because I think that the presence of what I felt were likely AI-obtained solutions on the leaderboards / on the Eulerians list really harms the legitimacy of those lists, and cheapens the accomplishment of those who actually solved the problems themselves. In retrospect, I'm not sure whether posting about this semi-publicly, instead of directly conveying my concerns to the PE admins, was the correct thing to do -- on the one hand, I think that it's good to have the concerns out in the open, rather than just simmering and invisibly discouraging people; on the other, I don't think that court-of-public-opinion is a great way to actually handle this, and looking back it seems like my post may have come across as antagonistic to the PE admins for not having already dealt with it.


On the object level, I think that in one of the cases, at least, my suspicions have been vindicated by the next problem (971) -- some of the early solvers like neverforget and andrew_gu posted in the solvers' discord thread that the original post by AlperTheKing had so much code and text that it was implausible that it was written by a human in the amount of time before that user submitted it, plus a number of tells (including unusual unicode characters), and was later greatly edited down to the version visible now.

While I'm sure that there are some users who are merely misunderstanding what kind of AI usage is permitted, I'm pretty skeptical that anyone who is using AI to get to the top of fastest solvers lists could be confused about the matter. It's no mystery that math and programming competitions (probably the closest things to PE) don't allow the kind of AI usage that would be necessary to do so, and the fact that "give the problem to the LLM" is enough to get a highly-ranked time on many problems should be an obvious tell that this is not intended behavior.

My personal feeling is that if community policing to deal with this becomes too onerous, it would be better to have no global Eulerians / FS lists than to have lists where AI-cheaters are outcompeting the vast majority of even very strong legitimate solvers. Others may disagree, but I feel like being beaten out by cheaters is much more discouraging than being beaten by people who are simply faster solvers than me.

[mdean]

I'm guessing this will put one of my recent posts out of compliance? I found some C++ library that looked useful. I don't know what version of C++ my compiler uses, but it's apparently not C++ 20 as that library was not available. I asked CoPilot for the best way to implement one of the functions from said library and it involved some command specific to gcc. I ended up copying the 4 line function. It's clearly labeled what it is and where it's from. The rest of the code and the analysis are mine and it's not one of the 10 most recent problems. What would you suggest I do?

[hk]

To me this example is within what I would call honest.
No problem with me.
I would leave it as is

[euler]

What would you suggest I do?

I agree with Hans. I don't think what you have done comes even close to being an issue. As the preamble in the genAI article says, "There is a world of difference between obtaining the answer to a problem and learning how to solve a problem." And what you did only briefly touches on "learning how to solve a problem". The purpose of that statement was addressing the situation where someone works with AI to help them understand how to solve the problem, coming from having no idea at all. I don't see an issue with someone doing that. Whereas you, on the other hand, were simply using AI to assist you in implementing a library to fulfil a clearly defined purpose you already had in mind.

[yourmaths]

I really appreciate your adding this language about what uses of generative AI are acceptable (in particular making it clear that using it for learning about a topic is OK, but that having it tell you how to solve a problem or write the code for you is not).

I agree with universalset. It is good to have an official PE policy for use of AI.

I have found chatGPT, etc. pretty useful for answering questions like: "Can we use method A to solve problem B?" and "How does concept C relate to concept D?" followed by further questions to clarify my understanding (or lack thereof). I see this as equivalent to (but a lot faster!) going through a heap of books and websites which may or may not have the information that I am after.

Also I consider something like "write a bisection search function" acceptable, which, since I have done this by hand many times, and understand the concepts well enough, I am happy to hand over the implementation to genAI.

I find a succinct guideline to using AI is the comment on this year's Advent of Code FAQ: would you ask your friend to go to the gym in your place? If you don't understand what's going on in a problem, asking chatGPT to generate code is cheating.

[Incompleteusern]

On the object level, I think that in one of the cases, at least, my suspicions have been vindicated by the next problem (971) -- some of the early solvers like neverforget and andrew_gu posted in the solvers' discord thread that the original post by AlperTheKing had so much code and text that it was implausible that it was written by a human in the amount of time before that user submitted it, plus a number of tells (including unusual unicode characters), and was later greatly edited down to the version visible now.

Would like to mention that in my opinion AlperTheKing's code for 975 is likewise very unlikely to be written by a human and not an AI, once again due to its length and various idiosyncrasies that make little sense, especially with the context that this was once again the fastest solve.

[GNFS]

Why would a cheater voluntarily share his code? It's not like other competition programming sites where you must submit your source.

[oblivyawn]

Why would a cheater voluntarily share his code? It's not like other competition programming sites where you must submit your source.

A lot of people online create idealized fantasy versions of themselves they wish to live out and it boosts their self-esteem. I believe this is what's going on.

For example, cheating in online chess feels baffling (similar to PE). There is no monetary benefit and due to anonymity there is no status benefit either. So what could possibly be gained? A lot of chess moderators (chess.com/lichess) report that cheaters will say something akin to "I wanted to feel like I was as good as I think I am.”

I believe this is what is mostly going on with PE cheaters. They've told themselves they are good enough to be Eulerians (or fantasize about it) and within their online fantasy versions of themselves cheating gives them a self-esteem boost, if only temporary - the same way it gives those chess cheaters.

I believe sharing their code makes it feel more authentic to them.