For the last ten years we’ve been building and maintaining a database of which websites on the internet run on green energy, and when they switched, then making this data freely available to all. With some help from the nice folks at SIDN, we’ve built a new API, to allow partners to programmatically update this information about their digital infrastructure, and how it’s powered. Read on for more details, and how to use it.
We’ve offered a read API for a few years now at the Green Web Foundation. You use it by passing in a website address, and and it returns a green / grey result. It’s what we use under the hood when you try run a check on our main website.
Why we exposed an API – so others offer green website checks too
We exposed this as an API for a number of reasons. We used it ourselves in this green check as mentioned above, but it was also a good way for the browser extensions we offer to incorporate the same checking features, and ti turned out to be a pretty handy for tools like website carbon, and ecograder as well, amongst others.
When an API gets a bit too popular for comfort
We also learned that over time, the data we had been offering over the API ended up being used for academic research, and particularly in 2020, we started seeing some extremely heavy spikes in use of the API, as researchers ran checks against increasingly large lists of websites.
Sending millions of requests over the wire to requesting checks against a list of websites, one website at a time isn’t a very efficient way to do these checks, so earlier in February we released datasets.thegreenwebfoundation.org.
This offers a way for folks to query the same data we make available over the API, but express queries using a well known, battled tested language, SQL, and download the results of any query in as either JSON or in CSV form.
Hopefully this will help our poor API servers being hammered by researchers so much 🙂
A new way to write to the Green Web Foundation Database
In addition to having some new ways to query and download data, there’s now a new, API for providers to programmatically edit information instead of having to manually sign into an admin website to register new networks, IP addresses and so on.
At present, whenever a hosting provider moves to a new datacentre, or they register new IP addresses, they also need to update the green web foundation website for the changes to be reflected in search results. If you’re a customer of a given hosting provider, and you’re seeing a sad face instead of a happy green one, this is frequently the reason – the provider often hasn’t got round to updating the records yet.
With the new API , we’ve made it easier to stay up to date, and we now have a RESTful API, that exposes a number of endpoints to allow programatica updates.
Using Open API to make it easier to integrate with new services
We’ve based the new API around the Open API specification – which means we’re able to expose a nice new interactive API page like the one in the screenshot below, listing all the possible endpoints we make available:
There are some other nice benefits too though – if you’re signed into the admin site, you can also try out the API, to see what kind of responses you get back, when prototyping a new service that works with the data. Below is a screenshot showing the an example of the one of the new API features, some inline documentation, and examples of what kind of data is expected, along with the responses to expect when you have made a successful request.
Releasing an API is one thing, but you always learn new things as you see it being used, and we expect that we’ll need to make various changes as we see people using it more.
If you’re interested in trying it out the new partner api, the APi pages are visible at https://admin.thegreenwebfoundation.org/api-docs/.
Eating our own dogfood
To demonstrate using this new richer functionality, we’ve now added a ‘batch’ feature to our old API page – rather than needing to check pages individually, you can now upload a CSV file listing the domains you want checked, and you’ll get back a CSV file listing the results for each one. You can try it on our old page – https://api.thegreenwebfoundation.org/
