Maintaining large design-first API specs

API design tools and specifically OpenAPI + JSONSchema tooling has evolved dramatically over the last 12 months. In particular, I’ve been using speccy quite extensively as a tool to simplify the design process of my spec-first APIs.

Lets first dig into how we define REST APIs using OpenAPI spec and how big those definitions can become pretty quickly.

A common REST API

To over simplify things, broadly speaking, modern REST APIs consist of a few key things:

• URLs – A way to address or locate a given resource 
• Verbs – Actions that you complete on a given resource
• Resources – the serialised data representing a given domain object
  

In the OpenAPI v3 spec, the combination of all three of these is better known as an “operation” and is defined in a similarly hierarchical fashion. For example, take this simple API definition for a user:

...
paths:
  /pets:
    get:
      operationId: listPets
      responses:
        '200':
          description: A paged array of pets
          content:
            application/json:    
              schema:
                $ref: "#/components/schemas/Pets"
...

While it is possible to define the response inline, its generally considered best practice to define resources once and then $ref-erence them.

For example sake, lets assume that a resource is addressed via its /resource and /resource/:id URIs and has full HTTP verb support accordingly:

• (list) GET /resource
• (create) POST /resource
• (fetch) GET /resource/:id
  
• (upsert) PUT /resource/:id
• (update) PATCH /resource/:id
• (remove) DELETE /resource/:id

For an API of just a handful of simple resources, the full API spec could grow long and unwieldy very very quickly. A simple library API with users books and loans could be almost 20 operations and hundreds of lines of YAML.

In my view, hundreds of lines of YAML isn’t very maintainable. And that’s where speccy comes in.

Logical endpoint groupings

As APIs grow, I start to group operations into logical endpoints where all operations share a given resource or domain process. To follow the library example, there would be a “user” endpoint, a “book” endpoint” and a “loan” endpoint.

Yet, OpenAPI has no concept of an endpoint. For disambiguation sake, some people consider operations and endpoints to be the same “thing” though for the purposes of this example I would disagree.

Speccy has a wonderful little feature that resolves external references between files. That means you can achieve the following API spec:

...
paths:
    # The /user endpoint
    '/user':
        $ref: "endpoints/user.yml#/paths/~1user"
    '/user/{id}':
        $ref: "endpoints/user.yml#/paths/~1user~1{id}"

    # The /book endpoint
    '/book':
        $ref: "endpoints/book.yml#/paths/~1book"
    '/book/{id}':
        $ref: "endpoints/book.yml#/paths/~1book~1{id}"

    # The /loan endpoint
    '/loan':
        $ref: "endpoints/loan.yml#/paths/~1loan"
    '/loan/{id}':
        $ref: "endpoints/loan.yml#/paths/~1loan~1{id}"
...

The root of my OpenAPI spec now becomes a sort of index for all the endpoints in my API which is a lot nicer and easier to manage. Alas, the JSONPath notation for JSON References is… unpleasant on the eye, it is clearly defined and I’ve certainly seen worse syntax in other places in tech.

Endpoints can now clearly be defined as a sort of map between the URI, verb and resources.  Here’s a snippet of the /user endpoint:

...
paths:
    '/user':
        get:
          operationId: getUserCollection
          tags:
            - Users
          parameters:
            - $ref: '../openapi.yaml#/components/parameters/filters'
          description: "Get a list of users"
          responses:
            '200':
              description: List of users
              content:
                application/json:
                    schema:
                      $ref: '../openapi.yml#/components/schemas/user-collection'
...

Admittedly this fragments the API definition and makes it substantially more difficult to use any existing API spec editors and UI’s. That, for the time being, is a trade-off I’m willing to make.

Resolving endpoints

Using speccy’s resolve functionality we can ‘combine’ these endpoints together into one publishable API spec:

speccy resolve library.yml > dist/library.yml

All going well, the output should be a nicely formatted, single file API spec that you can publish to consumers.

In the same way that some languages compile before packaging and distribution, speccy allows us to maintain a source to our API, before resolving and publishing it for consumers.

Standalone API definitions

Since the API specification requires some form of pre-processing, I store API specs as a standalone git repository, complete with branch/merge workflows, linting and testing. Here’s a snippet of a sample package.json for such purposes.

{
  "name": "library-spec-example",
  "version": "0.0.1",
  "description": "A library API specification",
  "scripts": {
    "test": "node_modules/.bin/speccy lint library.yml",
    "publish": "mkdir -p dist && node_modules/.bin/speccy resolve library.yml > dist/library.yml",
    "serve": "node_modules/.bin/speccy serve library.yml",
    "release": "cp dist/library.yml public/library.yml"
  },
  "devDependencies": {
    "speccy": "^0.8"
  },
...
}

For testing, I invoke npm run test, and for packaging and releasing I run npm run publish && npm run release .

With this in place, /public is a clean directory that I can use to serve my API spec to clients using my favourite static content hosting platform. My current favourite is Netlify – but that’s another blog!