How to implement JSON-API standard in MongoDB and Go

JSON rendering is a very easy task in Go. It is one thing to render JSON, it is another to set the structure to handle data, errors, links, metadata and other necessary information. JSON-API is one of the standards made to solve this issue and we are going to use it alongside our framework to create an API.

JSON doesn't force you to use any structure and without a structure (your own or an established one) you are sure to have some inconsistencies at one point in your API. A basic example, I often see plain text errors in JSON APIs and it's just wrong, the client is expecting a certain content type and the response is not what is expected. In addition HTTP statuses are often misinterpreted and 2 different APIs might end up using the same status for different types of errors resulting in headaches for the users. By using a standard, we don't spend time making our own structure and can focus on making user-friendly APIs.

This is the final article in the five-part series "Build Your Own Web Framework in Go":

The API

We are going to build an API to manage teas, a simple REST API to create, retrieve, update and delete teas. It can serve as a foundation for an mobile tasting application or to manage items in an eshop. And we are going to use MongoDB to not deal with DB schemas and migrations because it's not the subject of this article.

The example from the previous article (you can find a gist of the code here) already included a DB connection but was fake just for the purpose of the article, so let's replace it with a real DB now:

func main() {
  session, err := mgo.Dial("localhost")
  if err != nil {
    panic(err)
  }
  defer session.Close()
  session.SetMode(mgo.Monotonic, true)

  appC := appContext{session.DB("test")}
  commonHandlers := alice.New(context.ClearHandler, loggingHandler, recoverHandler)
  router := httprouter.New()
  router.Get("/teas/:id", commonHandlers.ThenFunc(appC.teaHandler))
  http.ListenAndServe(":8080", router)
}

We use mgo for the MongoDB driver, don't forget to get it and import it:

go get gopkg.in/mgo.v2

The handler to find a tea looked like that:

func (c *appContext) teaHandler(w http.ResponseWriter, r *http.Request) {
  params := context.Get(r, "params").(httprouter.Params)
  tea := getTea(c.db, params.ByName("id"))
  json.NewEncoder(w).Encode(tea)
}

It's a non-working example so let's make it workable.

Rendering JSON

Like I said above, rendering JSON is easy. We pass anything to the json package and it tries to convert it to JSON. The structure of a tea would be like:

type Tea struct {
  Id int `json:"id"`
  Name string `json:"name"`
  Category string `json:"category"`
}

JSON-API standard however is a bit less easy but it's not hard either. The JSON response must have a top-level key containing document and its name is the collection's name (teas in our case) but it can also be data so let's use data. This top-level element will have a individual tea as its value. It would translate to a simple struct like:

type TeaResource struct {
  Data Tea `json:"data"`
}

And we can now update our teaHandler to encode our found tea to the appropriate structure:

func (c *appContext) teaHandler(w http.ResponseWriter, r *http.Request) {
  params := context.Get(r, "params").(httprouter.Params)
  // tea := getTea(c.db, params.ByName("id"))
  w.Header.Set("Content-Type", "application/vnd.api+json") // We must set the appropriate Content-Type.
  json.NewEncoder(w).Encode(TeaResource{tea})
}

Finding the tea

To finish the handler, we need to replace the fake DB access by the real one. One way to do it is to create a repository that mediates between the entity TeaResource and the DB.

type TeaRepo struct {
  coll *mgo.Collection
}

func (r *TeaRepo) Find(id string) (TeaResource, error) {
  result := TeaResource{}
  err := r.coll.FindId(bson.ObjectIdHex(id)).One(&result.Data)
  if err != nil {
    return result, err
  }

  return result, nil
}

Since the id is a MongoDB ObjectId, we need to change the type in our Tea struct:

type Tea struct {
  Id       bson.ObjectId `json:"id" bson:"_id,omitempty"`
  Name     string        `json:"name"`
  Category string        `json:"category"`
}

Then we can replace the commented line in our handler by real code.

func (c *appContext) teaHandler(w http.ResponseWriter, r *http.Request) {
  params := context.Get(r, "params").(httprouter.Params)
  repo := TeaRepo{c.db.C("teas")}
  tea, err := repo.Find(params.ByName("id"))
  if err != nil {
    panic(err)
  }

  w.Header().Set("Content-Type", "application/vnd.api+json")
  json.NewEncoder(w).Encode(tea)
}

We now have a working handler. You can find the code here. recoverHandler will catch the panic and return an error but it's a plain text error, we must have errors that conform to JSON-API standard.

We could use curl to make test it:

curl http://localhost:8080/teas/123

{
  "data": {
    "id": "123",
    "name": "sencha",
    "category": "green"
  }
}

Errors

Errors in JSON-API are JSON documents with a top-level key named errors and a collection of errors. These errors can have a variety of information but the most important are: id, status, title, detail. The Go structs to represent errors would then be:

type Errors struct {
  Errors []*Error `json:"errors"` // We use an array of *Error to set common errors in variables. More on that later.
}

type Error struct {
  Id     string `json:"id"`
  Status int    `json:"status"`
  Title  string `json:"title"`
  Detail string `json:"detail"`
}

To return a response with JSON when panicking, we need to update our recoverHandler middleware.

func recoverHandler(next http.Handler) http.Handler {
  fn := func(w http.ResponseWriter, r *http.Request) {
    defer func() {
      if err := recover(); err != nil {
        log.Printf("panic: %+v", err)
        // http.Error(w, http.StatusText(500), 500) // What we had before
        jsonErr := &Error{"internal_server_error", 500, "Internal Server Error", "Something went wrong."}
        w.Header().Set("Content-Type", "application/vnd.api+json")
        w.WriteHeader(jsonErr.Status)
        json.NewEncoder(w).Encode(Errors{[]*Error{jsonErr}})
      }
    }()

    next.ServeHTTP(w, r)
  }

  return http.HandlerFunc(fn)
}

Accepting the right Content-Type

To return JSON-API responses, the client needs to accept to receive such responses. Requests must set Accept HTTP header with the Content-Type of JSON-API application/vnd.api+json. And our API needs to check if the header is set accordingly. To make that check on all our handlers, the easiest way to build it is as a middleware. Let's name it acceptHandler.

func acceptHandler(next http.Handler) http.Handler {
  fn := func(w http.ResponseWriter, r *http.Request) {
    // We send a JSON-API error if the Accept header does not have a valid value.
    if r.Header.Get("Accept") != "application/vnd.api+json" {
      jsonErr := &Error{"not_acceptable", 406, "Not Acceptable", "Accept header must be set to 'application/vnd.api+json'."}
      w.Header().Set("Content-Type", "application/vnd.api+json")
      w.WriteHeader(jsonErr.Status)
      json.NewEncoder(w).Encode(Errors{[]*Error{jsonErr}})
      return
    }

    next.ServeHTTP(w, r)
  }

  return http.HandlerFunc(fn)
}

func main() {
  // ...
  commonHandlers := alice.New(context.ClearHandler, loggingHandler, recoverHandler, acceptHandler)
  router := httprouter.New()
  router.Get("/teas/:id", commonHandlers.ThenFunc(appC.teaHandler))
  http.ListenAndServe(":8080", router)
}

To DRY up our error response code, we can create a function:

var (
  ErrNotAcceptable        = &Error{"not_acceptable", 406, "Not Acceptable", "Accept header must be set to 'application/vnd.api+json'."}
  ErrInternalServer       = &Error{"internal_server_error", 500, "Internal Server Error", "Something went wrong."}
)

func WriteError(w http.ResponseWriter, err *Error) {
  w.Header().Set("Content-Type", "application/vnd.api+json")
  w.WriteHeader(err.Status)
  json.NewEncoder(w).Encode(Errors{[]*Error{err}})
}

This way we can easily add errors in variables and use them wherever we want and we just have to call WriteError(w, ErrSomething) when we want to return an error response. You can checkout the code with this refactoring here.

The implementation of most of the standard for reading an individual resource is finally done. As you see it's not complex code but there are small details and we need to be careful when implementing a standard like JSON-API (but it applies for others too).

Create a tea

Now let's make an endpoint to create a new resource. First we add the route:

// We didn't have the function to wrap our handler for httprouter on POST requests
func (r *router) Post(path string, handler http.Handler) {
  r.POST(path, wrapHandler(handler))
}

func main() {
  // ...
  commonHandlers := alice.New(context.ClearHandler, loggingHandler, recoverHandler)
  router := httprouter.New()
  router.Get("/teas/:id", commonHandlers.ThenFunc(appC.teaHandler))
  router.Post("/teas", commonHandlers.ThenFunc(appC.createTeaHandler))
  http.ListenAndServe(":8080", router)
}

Then the handler:

func (c *appContext) createTeaHandler(w http.ResponseWriter, r *http.Request) {
  repo := TeaRepo{c.db.C("teas")}
  err := repo.Create(/* data to create a resource */)
  if err != nil {
    panic(err)
  }

  w.Header().Set("Content-Type", "application/vnd.api+json")
  w.WriteHeader(201) // When the resource is successfully created, we use 201 status.
  json.NewEncoder(w).Encode(body)
}

The structure is very similar to the find handler except we need a Create function on the repo.

func (r *TeaRepo) Create(tea *Tea) error {
  id := bson.NewObjectId()
  _, err := r.coll.UpsertId(id, tea)
  if err != nil {
    return err
  }

  tea.Id = id

  return nil
}

The only missing piece is the data to create the resource. These data are in the request body so let's read and parse it.

Parsing Request Body

With JSON-API, the body has the same structure as the responses, like this:

{
  "data": {
    "name": "phoenix",
    "category": "oolong"
  }
}

This is great because we won't need to create a new structure to handle these data. So we can create a function to decode the JSON body, then set the values into the context. Because this function will be useful for the update handler too, it is better to create a middleware to reuse the code across our API (and other projects in the future).

func bodyParserHandler(http.Handler) http.Handler {
  fn := func(w http.ResponseWriter, r *http.Request) {
    v := TeaResource{}
    err := json.NewDecoder(r.Body).Decode(&v)

    if err != nil {
      WriteError(w, ErrBadRequest)
      return
    }

    context.Set(r, "body", v)
    next.ServeHTTP(w, r)
  }

  return http.HandlerFunc(fn)
}

But using TeaResource inside the middleware is not very portable. We are not going to be able to use this middleware anywhere else. We need a way to make this more generic, but interface{} will not cut it because we will not be able to change it to TeaResource in our main handler.

To solve this issue, we need reflection. We will create a function taking any interface so when we call it with an empty TeaResource it will set the right value type with reflection and will return the middleware.

func bodyParserHandler(v interface{}) func(http.Handler) http.Handler {
  t := reflect.TypeOf(v)

  m := func(next http.Handler) http.Handler {
    fn := func(w http.ResponseWriter, r *http.Request) {
      val := reflect.New(t).Interface()
      err := json.NewDecoder(r.Body).Decode(val)

      if err != nil {
        WriteError(w, ErrBadRequest)
        return
      }

      context.Set(r, "body", val)
      next.ServeHTTP(w, r)
    }

    return http.HandlerFunc(fn)
  }

  return m
}

We then add the middleware in the middleware stack of the route.

router.Post("/teas", commonHandlers.Append(bodyParserHandler(TeaResource{})).ThenFunc(appC.createTeaHandler))

And finally we can use the body in our handler.

func (c *appContext) createTeaHandler(w http.ResponseWriter, r *http.Request) {
  body := context.Get(r, "body").(*TeaResource)
  repo := TeaRepo{c.db.C("teas")}
  err := repo.Create(&body.Data)
  if err != nil {
    panic(err)
  }

  w.Header().Set("Content-Type", "application/vnd.api+json")
  w.WriteHeader(201) // When the resource is successfully created, we use 201 status.  
  json.NewEncoder(w).Encode(body)
}

We are done, let's test it with curl:

curl -H 'Accept: application/vnd.api+json' -d '{"data":{"name":"phoenix","category":"oolong"}}' http://localhost:8080/teas

{
  "data": {
    "id": "456",
    "name": "phoenix",
    "category": "oolong"
  }
}

Since the request has to have a JSON-API body, the application would need to require the right Content-Type header, like it does with the Accept header. The middleware would only be for POST and PUT requests and be almost identical to the Accept header so I didn't included it in the article, but you can find it in this gist.

Other handlers

The other handlers don't have anything special and the article is already pretty long so I'm not going to write about them. But you can get the code here.

Wrapping up

JSON-API has many more features like links, metadata, compound documents and I will make sure to write more articles on the subject later. Still we have implemented all we needed to create a simple JSON API. When using Ember.js, no configuration would be necessary to use models with this API. And it applies for any library able to consume APIs conforming to JSON-API.

Even with more rules and structures, it remains easy to create JSON APIs in Go. And with all you learned, your API will be more production-ready than ever (don't forget to write tests though!).

Last but not least, I will be releasing a framework very soon (like next week if all goes according to plans). It will focus on making JSON APIs. This series of articles have been made while developing the framework, but there will also have features I didn't mention yet and it's really going to be awesome. Less work to make APIs, more focus on making great applications and happy users!