important information (Thu 27 Mar 08)

[stijn263]

Due to security issues, Project Euler moved recently to a new host. Our new hosts have been very helpful, providing proper security and support. However, it seems that Project Euler has finally outgrown shared hosting; CPU usage is exceeding our limit. Consequently, and in order to keep Project Euler online, I have had to take the regrettable decision to temporarily suspend certain key services on ALL accounts: the statistics page, the forum, and checking / submitting answers. You will still be able to view the problems, but until we can resolve this problem and find an alternative hosting plan it will need to remain this way. Sorry! )c:

Perhaps you can offer some advice in this matter? We are looking for a dedicated / semi-dedicated server option. Maybe you use a host provider that you could recommend? If you think you can help then please contact me: euler_help@mathschallenge.net.

That sounds pretty bad, I hope you can work it out..

* still looking for a donate button *

[hk]

* still looking for a donate button *

Me too.

[sfabriz]

Me too.

[rlindley]

count me in

[joshbowman205]

What are the costs of upgrading to a dedicated server option? I think there would be a lot of people willing to donate.

[ed_r]

Especially if we could have some of the nice stats back online ...

[sfabriz]

@admins:
along with a donate button I would suggest considering google ads on the page.
Maybe you won't get rich with those, but I guess they could provide some pounds a day.
Personally I wouldn't mind at all, if they help this wonderful website staying alive and healty.

[ansrivas]

I remember somebody suggested the idea of a donate button long time back but got no response. Is the PE team unwilling to take this step? Hope u guys can resolve the problem soon and we can get back to more PE fun

[markcrim]

it seems to me a natural fit would be a university comp sci department. good publicity for the univ, high bandwidth for you. i'm at a univ that is flexible about such things, but not in a relevant department....

can you tell me what kind of bandwidth the site uses?

oh, and if it's a matter of cpu, could we just buy them a server (computer)? they're cheap.

[sfabriz]

Stupid friday without p.e., that's so sad...

[euler]

@sfabriz: I know; I am so sorry!

it seems to me a natural fit would be a university comp sci department.

That would be perfect if we could find one.

I've already emailed the following information to a number of people who were interested, so rather than reinvent the wheel...

Here are the daily statistics for 26 March; the day we triggered our 48 hour account suspension warning. The reason we got a warning is because we've only just moved to our current hosts and the previous ones obviously didn't care too much about these statistics, as this was a typical day. In fact, maybe slightly lower than usual; certainly Friday/Saturday they are higher.

Hits = 92005
Files = 69301
Pages = 18993
Visits = 2999
Bandwidth = 588MB
CPU time = 2512.7s
Execution time = 37289.18s
Average memory = 2.58MB
Processes = 31293
MySQL queries = 78068

The bandwidth is not an issue, it's the CPU usage. They work on the basis that of the 24*60*60 = 86400 seconds in a day, 50% must be left free and 10% is reserved for Exim services. This means that calculations are based on 40% of 86400 = 34560 seconds. As we used 2512.7 seconds of CPU time on that day, it equates to 2512.7/34560 = 7.3% CPU usage; our limit is 4.5%.

It seems that these CPU restrictions are typical for shared hosting and is a figure that is usually hidden away in their lengthy service level agreement documentation. And where they don't have limits it means that one resource hungry customer could steal all the CPU time and compromise other customer's performance.

[sfabriz]

@euler: it's not your fault man!
I guess this weekend it will be chess rather than maths...

[Taifu]

* still looking for a donate button *

Definitely: me too!

[stijn263]

Does this mean that problem 188 won't be available for another week? If possible, I'd vote for publishing it anyway. Not being able to submit an answer yet would encourage me to take the time to program a decent solution

It looks like a donate button would generate quite a few pounds

[joshbowman205]

Im not sure its such a good idea, it might push up the cpu usage ( publishing an extra problem )

[BjornEdstrom]

There should be some time/space tradeoffs possible here.

http://projecteuler.net/index.php?section=problems <-- this can be made into a static html-page that is updated once a week. No PHP necessary to serve it, just the httpd. I don't know much about JavaScript, but if it's possible to read cookies with JS something like this can be coded:

When a solution is submitted, and it's the correct answer, the following happens:
1) The database is written. This is the only time the database is written (except for cases such as new user registrations).
2) A cookie is sent to the user. This cookie is set to all the problems the user has solved.
3) A javascript at the html version of http://projecteuler.net/index.php?section=problems reads the cookie and, on the client side, modifies the page to show problems solved/not solved.

A problem that must be solved on the server side is checking whether a submitted answer is right or wrong. This can also be solved without server side scripting, but the solution is quite dumb. The correct answer is a PHP script such as http://projecteuler.net/<correctanswer>.php. A custom 404-page is set to show "wrong answer". Did I mention the solution is dumb? In either case this should lower the CPU usage as the right/wrong decision is now on the httpd and not PHP.

Once per day a cronjob can generate the top1000 page. That will only take a few milliseconds / 24 hours.

I don't have a really good solution how to do the forums without lots of database writes and html-generation/caching (and as far as I know this is how it's done already). It should be possible to somehow limit the database reads ands server side language use.

Unfortunately web development is not my strong side, otherwise I would try to play around with the ideas above and see hot it works in practice. In either case I hope this works out.

[hk]

I have postponed problem 188.
If you want to reduce serverload while Colin works on a solution for the current problem you can download a pdf with all problems up to #187 from my website: http://home.planet.nl/~hklein/problems/problems187.pdf

Perhaps a nice time to go back to some old problems and improve your solution?

[Tommy137]

I have postponed problem 188.

So I do have to look again at P184 for a whole week....

[Dogun]

Well, I'm up for donating. I think everybody who's done 30 problems or more here (and isn't a math guru) has learned something and enjoyed this place and recognizes that it's a valuable resource the needs to keep running.

And I can see why you might be averse to the javascript+cookie stuff. I'm curious what's ultimately causing all the expense - process startup at roughly 800kcycles makes that a very minor cost. I notice in the source for the problem index, the problems section is sort of strung together in one blob, making me think you've already devoted some thought to perf - not that that is a significant win.

Missed caching opportunities and any inefficient SQL queries look like the next most likely culprits? Or is 130ms cpu time/page load considered fairly reasonable?

[BjornEdstrom]

And I can see why you might be averse to the javascript+cookie stuff.

Why? This is the old school proven solution but with a little aesthetic twist for the 21:st centuary You can of course do without javascript and cookies altogheter, but that's the only way to create the illusion of being "logged in", without the server having to maintain sessions or generate anything user specific. In the 90's it used to be fairly common to have plenty of generated static HTML pages and a single "write script" in cgi-bin. Having the write script also write a cookie to the client is useful for display purposes. Note that the cookie will of course have no purpose other than display the little green/red icons, to make life easier for the user.

Some kind of profiling would be really nice, but that's probably difficult for shared hosting. My guess is session/user maintenance and displaying the problems page is the major culprit, but we all know how difficult it is to guess about these things.