Friday, August 29, 2014

How to create a Web API no one wants to use. Part 3: HTTP Status Codes

Introduction to this blog series


Later this year, Netflix will be closing their public APIs. Twitter and Google have already restricted their APIs. Despite prominent tech-companies making drastic changes to their APIs, the number of APIs keeps growing steadily. Suddenly, everyone has a Web API. Your car has one, Chuck Norris has one, and worst of all, tech-companies with no focus on quality whatsoever has one. And who are using these APIs? Most likely, no one.
This blog series will take you through the pitfalls of creating a Web API. From a developers perspective, what mistakes will result in no one wanting to use your API?

Part 1: URI Design
Part 2: HTTP Verbs
Part 3: HTTP Status Codes
Part 4: Result formatting
Part 5: Versioning
Exercises

HTTP Status Codes

When working with a public Web API, your greatest friend is Fiddler or Postman. Using those, you can debug your web requests and the responses given to you in return: 
The first thing any developer does when they issue a web request against an API is to check the HTTP Status Code of the request. Is the result a 200 OK? Yes, awesome! 
If you find pleasure in giving developers false hope, you should make your Web API return a 200 OK, even though the request was not successful. These types of "errors in disguise" will make developers flee from your API in no time. Saying "Your request was successful! ... Just kidding, gotcha!" is not a great way to attract users.  
Example: The World Bank API

In part 1 of this blog series, we had a look at the URI design of the The World Bank API. Let's go back to that API to see how that are handling their HTTP Status Codes. 
First, ask for all information they have on the country of Brazil: 
GET http://api.worldbank.org/countries/br?format=json 200 OK
As this is a valid request, all is good so far. Now, what happens if we specify a countrycode that does not exists, such as 'a'?
GET http://api.worldbank.org/countries/a?format=json 200 OK
That also returns a 200 OK, so maybe we were mistaken. Maybe 'a' is a valid country code afterall? Let's take a look at the response just to be sure: 
[{"message":[{"id":"120","key":"Parameter 'country' has an invalid value","value":"The provided parameter value is not valid"}]}]
Now, that's just mean. Let's try another one. What happens if we specify the format with a colon instead of an equal sign?
GET http://api.worldbank.org/topic?format:json 500 Internal Server Error
That's better, at least we're not given a 200 OK. We're given a 500 Internal Server Error, and they are even giving us a detailed outline of what is wrong: 
{"fault":{"faultstring":"Evaluation of script AssignQueryParamsCaseInsensitively.py (py) failed with reason: \"TypeError: cannot concatenate 'unicode' and 'NoneType' objects\"","detail":{"errorcode":"steps.script.ScriptEvaluationFailed"}}}
Problem: The World Bank API

The first issue here is that the API is returning 200 OK statuses despite there being something wrong with the request. The HTTP/1.1 protocol clearly states that 200 OK should be used when "The request has succeeded". 
The main issue, however, arises when we supply the incorrect format. The request produces a 500 Internal Server Error, although what really caused the issue was us supplying an invalid request. To make matters worst, the result contains a detailed error message. Displaying the internals of your Web API to the public makes it vulnerable to attack, and you should never ever return a stacktrace of the errors produced. 
Solution: The World Bank API

When you're developing a Web API, you should at least use the three most common HTTP Status Codes: 
Status CodeDescriptionFor dummies
200 OKThe request has succeededAwesome!
400 Bad RequestThe request could not be understood by the server due to malformed syntaxThe user of the API made a mistake
500 Internal Server ErrorThe server encountered an unexpected condition which prevented it from fulfilling the request.The creator of the API made a mistake
In addition to these, if you're serious about your API, you should also take a look at 201 Created, 304 Not Modified, 401 Unauthorized and 404 Not Found.
In the examples above, the request where we specified an invalid country code should have given us a 404 Not Found:
GET http://api.worldbank.org/countries/a?format=json 404 Not found

Specifying an invalid format should not have produced a 500 Internal Server Error, it should have given us a 400 Bad Request (without displaying its stacktrace):
GET http://api.worldbank.org/topic?format:json 400 Bad Request

Summary


HTTP Status Codes exist so that users of your API immediately know whether or not the request they made was successful. In the cases where the request was not successful, the status codes gives them a clear overview of why it failed. Disguising your errors as 200 OK's will only help in building up irritation, so you should consider using at the most common status codes when building your API. Also, do yourself a favour, and care about your API from the very first piece of code you write. 

Multiple CustomerContacts created in EPiServer Commerce

Creating a CustomerContact from a MembershipUser is something you'll run into quite quick when developing a Commerce site. Now, what's wrong with this piece of code?

public CustomerContact CreateUser(MembershipUser user, string firstname, string lastname)
{
   CustomerContact customerContact = CustomerContact.CreateInstance(user);

   SecurityContext.Current.AssignUserToGlobalRole(user, AppRoles.EveryoneRole);
   SecurityContext.Current.AssignUserToGlobalRole(user, AppRoles.RegisteredRole);

   customerContact.FullName = string.Join(" ", firstname, lastname);
   customerContact.FirstName = firstname;
   customerContact.LastName = lastname;
   customerContact.Email = user.Email;
   return customerContact;
}

To me, this piece of code looked fine until I ran it. What happened was that I received two CustomerContacts, one containing the correct information such as firstname, lastname and fullname. The other CustomerContact only contained the email address, nothing else. Both of them were linked to the correct account.

So what happened? Well, according to the EPiServer Commerce SDK we need to save the CustomerContact before calling AssignUserToGlobalRole"The CustomerContact associated with the user must be saved before calling this method. Otherwise contact information might be lost." 

In this case, contact information was not lost, it was duplicated. So what's the correct way of doing it then? We have to call SaveChanges on the CustomerContact before assigning the roles:

public CustomerContact CreateUser(MembershipUser user, string firstname, string lastname)
{
   CustomerContact customerContact = CustomerContact.CreateInstance(user);
   customerContact.SaveChanges();
   
   SecurityContext.Current.AssignUserToGlobalRole(user, AppRoles.EveryoneRole);
   SecurityContext.Current.AssignUserToGlobalRole(user, AppRoles.RegisteredRole);

   customerContact.FullName = string.Join(" ", firstname, lastname);
   customerContact.FirstName = firstname;
   customerContact.LastName = lastname;
   customerContact.Email = user.Email;
   return customerContact;
}

Thursday, August 28, 2014

How to create a Web API no one wants to use. Part 2: HTTP Verbs

Introduction to this blog series


Later this year, Netflix will be closing their public APIs. Twitter and Google have already restricted their APIs. Despite prominent tech-companies making drastic changes to their APIs, the number of APIs keeps growing steadily. Suddenly, everyone has a Web API. Your car has one, Chuck Norris has one, and worst of all, tech-companies with no focus on quality whatsoever has one. And who are using these APIs? Most likely, no one.
This blog series will take you through the pitfalls of creating a Web API. From a developers perspective, what mistakes will result in no one wanting to use your API?

Part 1: URI Design
Part 2: HTTP Verbs
Part 3: HTTP Status Codes
Part 4: Result formatting
Part 5: Versioning
Exercises

HTTP Verbs

There's an old story called "The Spider of Doom" where a company developing a content management system suddenly found that their client had "magically" lost all the pages of their website. After a lot of searching around, they found that the Googlebot was able to bypass their authentication mechanisms and had therefore crawled all their "Delete page" links. This, of course, resulted in all the pages of the website being deleted.

Imagine for a minute that you're creating a Web API that doesn't require authentication, but where the users should be able to delete content. You also want the endpoints to be crawled by search bots. How do you ensure that the bot doesn't delete all your content? Well, you make sure that any deletion of resources isn't done with a HTTP GET request. In other words, you use HTTP verbs in the way they were intended.

HTTP verbs makes it easy for other developers to consume your Web APIs without you having to manage and maintain several endpoints for each possible action. Sadly though, a lot of APIs only use the GET verb, and they use it wrong. On the bright side, that's a pretty great way of making sure no one will use your API!
Example: The Tesla Model S JSON API
I like to call the Testa Model S API the Batmobile API, and it pains me that I'm able to find something wrong with it. Tesla is only the greatest car ever built, and creating an API for it is genious, it only makes me want it more.
Let's start out by getting the current climate state of the car: 
GET /vehicles/{id}/command/climate_state
So far, so good. Remember, GET, PUT and DELETE are idempotent, meaning we can issue a request against them as many times as we want without changing the result. In addition to this, the GET method is safe, meaning it should not have any side effects or change state in any way. If we apply these two rules to the climate_state endpoint, we see that we can issue the request twenty times and we will always receive the climate state of the car in return, without changing the state of the car. So it clearly satisfies the criteria of the GET method.

Now, let's open the sun roof of the car:
GET /vehicles/{id}/command/sun_roof_control?state=open

Here we have a problem. When we open the sun roof of the car, we're changing the state of the car. Therefore, the sun_roof_control endpoint does not satisfy the criteria of the GET method.

Problem: The Tesla Model S JSON API

The main issue with this API is that it calls itself RESTful, but it uses the GET verb for all its endpoints except one. Had the API been implemented according to the HTTP protocol, then the following endpoint should have given us the current state of the sun roof, for example "open" or "closed". It should not change the state of the car in any way.
GET /vehicles/{id}/command/sun_roof_control

Solution: The Tesla Model S JSON API

In order to change the state of the sun roof, we should issue a PUT instead of a GET:
PUT /vehicles/{id}/command/sun_roof_control?state=open

The PUT method is not safe, meaning the request is allowed to change the state of the car. It is, however, idempotent. This means that the request above will always result in the sun roof being opened, whether we issue it two times or twenty. By using PUT, we're now satisfying the critera of the HTTP protocol.

What about POST and DELETE? Well, we can't delete sun roof, so the DELETE verb will not be supported. POST is usually used for creating items, and unfortunately creating a new sun roof is not allowed either.

GET (read)POST (create)PUT (update)DELETE (delete)
Returns the state of the sun roof (open/closed)Not supported, cannot create new sun roofChanges state of sun roof (open/close)Not supported, cannot delete sun roof

There is an extremely entertaining thread on Reddit about the Tesla Model S API and how honk_horn should have been implemented. I highly recommend taking a look at it!

Summary


If you're using the GET verb for all your endpoints even though state is changed, you will make developers using your API wrinkle their noses. Use the correct verbs for the job and you will see that maintaining your API will also be a lot easier. Also, do yourself a favour, and care about your API from the very first piece of code you write. 


Wednesday, August 27, 2014

How to create a Web API no one wants to use. Part 1: URI design

Introduction to this blog series


Later this year, Netflix will be closing their public APIs. Twitter and Google have already restricted their APIs. Despite prominent tech-companies making drastic changes to their APIs, the number of APIs keeps growing steadily. Suddenly, everyone has a Web API. Your car has one, Chuck Norris has one, and worst of all, tech-companies with no focus on quality whatsoever has one. And who are using these APIs? Most likely, no one.
This blog series will take you through the pitfalls of creating a Web API. From a developers perspective, what mistakes will result in no one wanting to use your API?

Part 1: URI Design
Part 2: HTTP Verbs
Part 3: HTTP Status Codes
Part 4: Result formatting
Part 5: Versioning
Exercises

URI design


If you're interested in creating a terrible Web API that will give developers headaches for ages to come and confuse them more for every piece of your API they use, you should start with your URI design. Badly named, inconsistent URIs that all violate the Principle of Least Astonishment will make your API just as disappointing as the movie "Mega Shark vs Giant Octopus".

Note: If you're building your API using hypermedia, these guidelines might not be applicable to you.

Example: U.S. City and County Web Data API

Starting out with an example, let's take a look at the "U.S. City and County Web Data API" created by The U.S. Small Business Administration.

Get all URLs they have registered for the city of Chicaco, Illinois:
GET http://api.sba.gov/geodata/all_links_for_city_of/chicago/il.json 200 OK

Assuming they use the same structure for all their endpoints, we can get all URLs they have registered for Cook Country, Illinois:
GET http://api.sba.gov/geodata/all_links_for_county_of/cook%20county/il.json 200 OK

So far, so good. What if I want to get all URLs they have registered for the state of Illinois? Based on their previous endpoints, this should work:
GET http://api.sba.gov/geodata/all_links_for_state_of/il.json 400 Bad Request

That didn't work. Which means that the developer using their API must head over to the documentation to figure out how it can be done, and the annoyance has already begun.

Problem: U.S. City and County Web Data API

The first thing to point out in this API is the terrible naming, which will at some point make this API difficult for the creators to maintain.

Using a structure such as all_links_for means you will end up with lots and lots of endpoints when the API grows. An example of this is already present in the API, where you should be able to retrieve not all links, but instead only primary links, which means all your endpoints will be duplicated:

http://api.sba.gov/geodata/all_links_for_city_of/...
http://api.sba.gov/geodata/all_links_for_county_of/...
http://api.sba.gov/geodata/primary_links_for_city_of/...
http://api.sba.gov/geodata/primary_links_for_county_of/...

Imagine what happens when they want to be able to give you broken links or secondary links, they will have to duplicate their endpoints for each new type of link, which in the end will be impossible to maintain.

The second problem with this URI structure is that it is not logical. As a developer, I am not able to predict what the URI of the next endpoint I'd like to use is based on my experience with the other endpoints.

Solution: U.S. City and County Web Data API

One of the many alternative ways this API could have been implemented. The pros of this implementation are:
  1. The URIs are readable and simple to understand
  2. The URIs are hackable, meaning I can remove the last part of the URI and still have a valid request.
  3. Querystring parameters are used for filters, so you don't have to duplicate all your endpoints when you'd like to add a new type of link.
Get all links for the state of Illinois:
/links/il

Get all links for Cook County, Illinois
/links/il/cook%20county

Get all links for Chicago, Illinois
/links/il/cook%20county/chicago

Get all primary links for Chicago, Illinois
/links/il/cook%20county/chicago?type=primary

Example: The World Bank API

Let's have a look at another Web API, The World Bank API. First, get all topics and indicators respectively:
GET http://api.worldbank.org/topics 200 OK
GET http://api.worldbank.org/indicators 200 OK

That looks like a pretty decent API, right? It's simple and it's logical. Now, let's get the topic with ID=5, and then get the indicators for this topic:
GET http://api.worldbank.org/topic/5 200 OK
GET http://api.worldbank.org/topic/5/indicators 404 Not Found

Ah, damn it. Logic no longer applies. In order to get all indicators for topic with ID=5, I can no longer use the plural name for indicator, so let's try with the singular name:
GET http://api.worldbank.org/topic/5/indicator 200 OK

Awesome, inconsistency!

Problem: The World Bank API

This API is not actually that bad, there are a lot worse of there. It follows many of the best practices out there, such as using nouns instead of verbs and having a logical structure.

However, the inconsistency between plural names (indicators) and singular names (indicator) strictly violates the Principle of Least Astonishment and makes the API inconsistent.

Solution: The World Bank API

Stick to plural nouns all the way in order to make the API consistent.

Get topics:
/topics

Get topic with ID=5:
/topics/5

Get indicators for topic with ID=5:
/topics/5/indicators

Summary


Badly named, inconsistent URI designs will not only be a pain in the arse for the developers using the Web APIs. They will also turn out to be a pain in the arse for the developers maintaining them. So do yourself a favour, and care about your API from the very first piece of code you write. 

Monday, August 11, 2014

How the media sexualized the story of a 16-year old girl in tech instead of acknowledging her work

Imagine you're a journalist. And you've been told to write a story.

You've been told to write a story on the winner of the 2013 Science in Action award. The winner is Elif Bilgin, a 16-year old girl from Turkey. The award is part of the Google Science Fair, and the title of her project is: "Going Bananas!-Using Banana Peels in the Production of Bio-Plastic As A Replacement of the Traditional Petroleum Based Plastic".

What do you do? Well, let's sum up the keywords: Bio-Plastic. Female. Google. 16-year old. Petroleum. Award winner. Tech. 

It sounds like a pretty great story, doesn't it? I think so. As a journalist, you have the chance to write a story that will inspire other youngsters who are interested in science. You have the chance to show the world that young girls with great ideas can be seen, acknowledged and appreciated for what they are doing. You have the chance to tribute women who are interested in technology.

Or, you could look at the story and think: Here's a girl working with bananas, I think we need to sexualize this. And that is exactly what the reporter at metro.se did when he wrote an article on the subject last week:


In english, the title reads: "You can never imagine what she can do with a banana". So, instead of attracting readers who would be interesting in reading about the great accomplishments of a 16-year old girl, the author decided to attract a completely different audience by sexualizing the article.
This pisses me off. First of all, the girl is not even an adult, she should not have to deal with men drawing parallels between her work and sex. Second of all, what she has done is incredible and the journalist shows no respect for what she has accomplished.

As a woman in tech, I glow with pride whenever I see other women do something that might attract even more women to our industry.

When I was younger, my mother would save any newspaper articles for me that could inspire me to pursue my dreams. I loved reading about other teenagers, both male and female, researching topics that could make a difference. Elif Bilgin would have been a true inspiration to 16-year old me, even more so than she is now. Would my mother have saved this newspaper article for me though? No, I don't think so. I think she would have wanted to spare me the insight of how some men decide to view women in tech.

I hope this journalist has learned his lesson. And the next time he gets the chance, I hope he decides to show the world that young girls with great ideas can be seen and appreciated for what they are doing.

If you'd like to read a serious article on Elif Bilgin's project, I recommend taking a look at this blog post from Scientific American.

Note: Metro.se decided to change the title of the article after receiving numerous complaints. You can read the article here.