Open Exchange Rates update – The party’s not over, it just got a little too noisy

Update 11-07-12: App ID signup is available for Open Exchange Rates. Check it out!

Update 24-06-12: All the data have now been removed from the GitHub repository. This GitHub issue has the details!

I’ll keep this post updated as the news comes in, so hit back here or follow me on Twitter.

I’ve spent the past week grinding on a relaunch of Open Exchange Rates, designed mostly to prevent certain specific people – of dubious coding ability – from hammering the server up to 30,000 times a day (each!)

As it happens, those few have also been turning some heads at GitHub, where the Open Exchange Rates open-source repository is hosted: (emphasis mine)

Hi Joss,

I came across your repo while analyzing some performance problems on GitHub.com – cool site! However, some people didn’t read your great API docs, and are crawling your GitHub repo directly instead of hitting up the website. This is causing our server quite a bit of stress and is impacting other users.

Can you remove the ‘historical’ and ‘latest.json’ files from the GH repo and only host the code here? Like I mentioned, if people were following your instructions this wouldn’t be a problem (JSON is small!), but someone is accessing this data in a very dumb way in a tight loop.

Before anything else, I’d like to give a huge thanks to the team at GitHub, for allowing Open Exchange Rates to take off in the way it has. Without GitHub, it simply wouldn’t exist today – nor would CurrencyBot, for that matter.

They’ve been awesome about the project and understanding about its aims, but at the end of the day, I don’t want to be causing them any hassles.

Where we go from here:

This is as good a time as any to introduce the changes coming to the project, now amended to include today’s update:

“Open” Exchange Rates

• The data – hourly-updated and historical exchange rates back to 1999, plus other bits – will still be available from openexchangerates.org as before (it will always be available free
• You can access the data at http://openexchangerates.org/latest.json, but starting next week, you’ll need to grab an API key (see below – this is to prevent certain people from overloading the server)
• The service is also getting expanded right now, to include many new essential and lovable features, including more data, advanced queries and SSL support.

Git Repository

• The git repository will be removed from GitHub, but remain available (and updated) at openexchangerates.org, so you’ll still be able to grab it from there, as needed.
• I’ll post details about that once it’s available – I just need to figure out how to make it publicly clone-able (answers on a postcard!)

API Keys

• To prevent irresponsible coders from bringing my server (provided by the lovely TSOHost) to its knees, I’ll be giving out API Keys, which will be needed to query the service.
• Signup will be an easy, 60-second affair, and it’ll be just as simple to use the service – a matter of adding e.g. ?app_id=12345 to your requests.

Free and Paid

• Like Akismet, API Keys will be free (or “pay-what-you-like”) for personal/small scale users, and dirt-cheap for business and enterprise users.
• A lot of work and maintenance go into providing and maintaining a free service like this, so if you choose to pay a little bit to use the service, I’ll send you a personal thanks!

Exciting times ahead, exciting and busy times. Any questions, please hit me up on Twitter or email joss@openexchangerates.org.