Building Modern REST APIs with OpenAPI

No, I'm not talking about 'OpenAI', though you certainly can use it to assist in your API designs. I'm talking about the OpenAPI standard, a modern spec for defining REST APIs.

If you're reading this, I probably don't need to tell you that REST APIs are ubiquitous in tech. Practically every company has at least one, whether it be an internal or customer facing API. While other types of APIs have been gaining traction the last few years (GraphQL, gRPC, etc), REST is here to stay.

REST APIs offer a lot of advantages over other flavors. Namely, they're simple to set up and consume. A plain old cURL request can be used to interact with a REST API, and rarely requires any special tooling. However, they lack one huge benefit that GraphQL and gRPC have built into them: schemas. Without a schema file, we're left to the documentation provided by the developer, and no guarantees on what each endpoint expects or returns.

Challenges of Managing REST APIs

At Cycle.io, we've got one super beefy API with over 200 endpoints, and a few smaller specialized ones. These are REST APIs, and for good reason. Given the wide range of applications built on the Cycle platform, our API needs to be as simple as possible to consume. gRPC and GraphQL, while nice for internal use, were not viable for most of our clients, so we've stuck with REST for all public-facing APIs. But, in order to continue growing and improving our offering, we needed to solve some pain points.

Up until last year, we had been manually writing documentation and building language specific clients. These cost a lot of engineering time to build and were highly error prone - a change to docs may not always make it into every client in every language we wanted to support. It also caused a schism between platform and portal implementations - without a spec, the portal team couldn't easily add new features at the same time the platform team did. Instead, the platform team had to add the new functionality, log all the additions or changes to the API, add them to documentation, while the portal team needed to spend time updating the Typescript client (ensuring it was accurate) before they could even begin updating the interface to take advantage of the new endpoint. Our clients utilizing the API had to wait for these updates to make it downstream, further delaying their adoption of new features.

All this work to add a simple feature was tedious. Inspired by some gRPC & GraphQL schema-based workflows I'd seen, I set out to find a solution that would allow us to do API-first development on our REST API. That's when I discovered the OpenAPI standard.

What is OpenAPI?

On their website, OpenAPI Specification (OAS) describes itself as "a consistent means to carry information through each stage of the API lifecycle. It is a specification language for HTTP APIs that defines structure and syntax in a way that is not wedded to the programming language the API is created in."

Really, it's a contract. Utilizing a schema-first approach, engineers are able to plan and design the API, discover requirements that may have been hidden before, and generate documentation and language-specific servers and clients easily.

In practice, it looks something like this:

openapi: 3.1.0 info: title: Sample API description: Optional multiline or single-line description in [CommonMark](http://commonmark.org/help/) or HTML. version: 0.1.9 servers: - url: http://api.example.com/v1 description: Optional server description, e.g. Main (production) server - url: http://staging-api.example.com description: Optional server description, e.g. Internal staging server for testing paths: /users: get: summary: Returns a list of users. description: Optional extended description in CommonMark or HTML. responses: '200': # status code description: A JSON array of user names content: application/json: schema: type: array items: type: string

The appeal to me was instant. Once we retroactively documented our existing APIs into OAS, we could go forward by defining our API here, generate clients, and simultaneously build the endpoint into the server AND make the interface changes all at the same time. What's more, documentation would come with it for 'free'.

This also reduces friction for customers that consume our API. Instead of waiting for us to write or update downstream clients to get access to the latest features, they could generate their own using the latest spec in their language of choice, and cross reference with the hyper up-to-date documentation.

Quick Note: OpenAPI 3.1.0 vs 3.0.3 vs Swagger 2.0?

If you've looked at OAS at all, you've probably been confused on the differences in versioning and naming. I sure was.

To make it simple, OpenAPI is the name given to the Swagger 2.0 specification after it was donated by SmartBear to the OpenAPI Initiative and released as version 3.0.0. Swagger is still around but is generally referring to tooling around the OAS.

Though OpenAPI 3.1.0 has been out for over 3 years now (2021), a lot of the ecosystem is still catching up. There has been a noticeable increase in interest over the last couple years, and finally tooling is starting to update to support the latest version. OpenAPI 3.1.0 offers one significant advantage over its predecessors - it's compatible with JSON Schema. That means all your API models have double value and can be used with JSON Schema tooling. At Cycle, we take advantage of this by exporting our Stack file (like a docker compose file for our platform) as a JSON Schema that can be loaded into IDEs for linting/validation. Double the value, from one spec.

5 Tips for Writing OpenAPI Files

Without diving into too much detail on the specifics of OAS (see their docs for the hierarchy of a spec file), there are some tricks we learned that I think most people will find useful as they begin this journey.

1. OpenAPI files can be written in JSON or YAML. If you're planning to maintain your spec by hand like we chose, YAML is a decent option for keeping it human readable.
2. Most tools today require a single, stitched-together schema file to work. Of course, the bigger the API, the more unwieldy that file becomes. If we kept our massive spec as a single file, it'd be over 30,000 lines long. By utilizing tools like Redocly CLI, you can reference other files to 'inline' them via the $ref keyword. Normally, these are JSON pointers within the same file, but Redocly and other tooling allows for referencing external files and URLs, making it easy to split your spec into multiple files. For example:
   responses: 200: description: Returns a Container. content: application/json: schema: type: object required: - data properties: data: $ref: ../../../components/schemas/containers/Container.yml includes: $ref: ../../../components/schemas/containers/ContainerIncludes.yml
3. Come up with a plan on how you'll handle null AND omittable values. For Cycle, we decided to treat them the same, so all nullable fields can also be omitted (not listed in the 'required' field).
4. Use the 3.1.0 version of the spec. While nearly identical to 3.0.3, it allows for interop with JSON schema and using type arrays, so an item can be marked both i.e. a string and "null", without needing to use the 'nullable' keyword and the awkwardness of merging it with `$ref`. It's fairly easy to down-convert in case a particular tool or library requires the older version of the spec (though this is quickly changing).
5. Utilize `oneOf` and the `mapping` field to build discriminated union types (a OR b) for better documentation and clients: requestBody: description: Parameters for creating a new container job. content: application/json: schema: discriminator: propertyName: action mapping: start: ../../../components/schemas/containers/taskActions/ContainerStartAction.yml stop: ../../../components/schemas/containers/taskActions/ContainerStopAction.yml reconfigure: ../../../components/schemas/containers/taskActions/ReconfigureContainer.yml volumes.reconfigure: ../../../components/schemas/containers/taskActions/ReconfigureVolumes.yml reimage: ../../../components/schemas/containers/taskActions/Reimage.yml scale: ../../../components/schemas/containers/taskActions/Scale.yml oneOf: - $ref: ../../../components/schemas/containers/taskActions/ContainerStartAction.yml - $ref: ../../../components/schemas/containers/taskActions/ContainerStopAction.yml - $ref: ../../../components/schemas/containers/taskActions/ReconfigureContainer.yml - $ref: ../../../components/schemas/containers/taskActions/ReconfigureVolumes.yml - $ref: ../../../components/schemas/containers/taskActions/Reimage.yml - $ref: ../../../components/schemas/containers/taskActions/Scale.yml

Generating Client And Server Implementations

Once your spec is in place, it's easy to generate language-specific clients, or even stub out a server for quick iterations while conforming to the spec. Whether you're using them for internal development, or providing them for your customers, it will speed up integration and rollout of new features while guaranteeing compatibility throughout your products and ecosystem.

At Cycle, we heavily utilize Go and Typescript. Here are the libraries we use today to generate from our spec:

Typescript: OpenAPI Typescript

When it comes to supporting open API with TS/JS, nothing beats this library. It combines ease of use with a minimal footprint, as well as being compatible with the latest 3.1.0 spec. You can see our npm-installable client here.

React/Redux: RTK-Query

We use the excellent RTK Query library in our portal. Combining codegen to generate React hooks and type definitions means our portal is always up to date with our API and eliminates invalid requests/responses for a more stable interface.

Go: Ogen

While it doesn't support 3.1.0 yet, they are iterating quickly. Ogen boasts full support while handling null/optional values and discriminated unions. Ogen can also be used to generate a server stub for implementing a spec. You can see our client based on ogen here.

Honorable Mention: libopenapi

While not a full-blown client generator, this low-level library is starting to provide an excellent base for building OpenAPI-related tooling in Go. With full 3.1.0 support, it's only a matter of time before other generators start taking advantage of this library.

To generate clients for other languages, check out https://openapi-generator.tech/docs/generators.

Generating Documentation

There are quite a few documentation generators and companies specializing in hosting said API documentation. Some are free, some paid, and some ridiculously expensive. It was difficult finding ones that properly support 3.1.0, and many don't even properly support older versions with more complex URL parameters or other weird issues. These are the top 3 we found.

Redocly (Free + Paid)

If your API spec is modestly sized, and you don't mind the Redocly branding, then their free version, Redoc, will probably suit your needs. They support OpenAPI 3.1.0, and have some of the best support for all features in the spec. You'll have to manage hosting it yourself, but with Cycle it will be a breeze ;).

The downside with Redoc is it generates a single page and isn't very performant for larger specs. On Cycle's Platform API Documentation, Redoc was crawling and caused huge performance issues, so if your spec is larger, you may want to try something else.

Their paid services tend to get expensive, quickly. For custom domains, it's over $300/month. However, the product is better as it generates multiple pages and removes all of their branding.

Scalar (Free + Paid)

This up-and-coming OpenAPI focused company has some exciting tooling and competes with Redocly and other UI builders like Stoplight.io. Their docs look modern and are fully customizable too.

Unlike Redocly, their free and paid documentation are the same, but they'll host it for a reasonable ($12/m) cost.

Speakeasy (Paid)

Speakeasy does a lot of tooling around documentation, SDK generation, and validation. Their tools look nice, but are pretty expensive. With Speakeasy, however, at least you get access to more than just documentation.

Takeaways

When it comes to building APIs, treating your schema as the contract between different parts of your application and customers leads to more efficiency and stability. While REST doesn't have schemas built into it like GraphQL or gRPC, OpenAPI fills that gap.

OpenAPI schemas can be used to increase parallelization across teams, keep up-to-date documentation for internal teams and customers, and generate client SDKs for interacting with your applications in any language. At Cycle, we are able to build more stable DevOps tooling across our stack by guaranteeing compatibility between our API and user interfaces. Cycle's API spec is built in the open, so feel free to check it out as an example of how to write and manage a large-scale API (also open to PRs!).

Build Your APIs on Cycle

If you're looking for a home to host your APIs and their documentation, and exhausted from trying to piece together a myriad of tools just to be able to deploy your applications to production, we'd love to chat! Cycle is the LowOps platform for building platforms, and can reduce or eliminate complexity while reducing your long term costs.

💡 Interested in trying the Cycle platform? Create your account today! Want to drop in and have a chat with the Cycle team? We'd love to have you join our public Cycle Slack community!