Cycle's New Interface, Part II: The Engineering Behind Cycle’s New Portal

In our last installment, we covered the myriad of new UI changes added to Cycle’s portal. In this part, we walk through five of the tough engineering choices made when developing the new interface, discussing the alternatives that were considered, and shining a light on some of the technology our engineering team utilizes today.

Part 2: The Engineering Behind Cycle’s New Portal

Whenever someone says that you need to rebuild a complex piece of software from scratch, it’s almost universally the wrong decision. Almost, however, doesn’t mean always. In the case of Cycle’s “old portal”, there was half a decade of technical debt, built on top of libraries that were no longer maintained or updated, with bolted-on modules built to support new features of Cycle that were never planned. While normally, you’d take some time and upgrade/rebuild things piece by piece, it was going to be far better for the future of our portal if we started over. This is mostly because, when the original was built, the frontend/web ecosystem was in disarray and a new framework was coming out seemingly every week. While we got it right with React, even the standards in the React ecosystem were in flux. Class based components, functional components, Redux, MobX, CSS-in-JS, Tailwind… the number of decisions you had to make and be right on were overwhelming.

Today, the shape-shifting has slowed down, and there are stable and mature winners in the frontend world. While decisions still have to be made, it’s much less worrisome that the choices we make today will be obsolete in the next 6-12 months. And now that Cycle itself is more mature, with the most fundamental and complex pieces stable, it’s far easier to engineer an interface that will be more capable of growing with it for the next 5+ years.

So, with that preamble out of the way, let’s walk through the same choices our engineering team has made over the last 10 months to bring to you our new and powerful portal. The options for each choice aren’t exhaustive, but are the main ones we looked at after some basic elimination.

Choice 1 - Bundling Tool

• Webpack
• ✓
  Vite
• Esbuild
• Parcel

Every web app needs a good bundling tool. Traditionally the de facto choice was webpack, but over the last few years of using it, we’ve found it bulky and awkward. Not to mention, it tends to be slow when compared to more modern tools. After some testing, we decided to go with Vite. It is modern and follows modern web principles, is extremely fast at cold-boot builds and hot reloading, and has a ton of active development. The configuration is dead simple, which cannot be said for webpack. Looking ahead, there are plugins for building PWAs, which we’ll discuss in the final entry of this series, and it has plugins for simplifying sourcemap uploads to Sentry for event monitoring.

All of these features put Vite at the top of our list, and has made development on new portal seamless.

Choice 2 - React or Not React?

• ✓
  React
• Not React (Vue, HTMX, Svelte, etc)

This was by far the easiest choice we made. There are a lot of opponents, die-hards, and fanatics in the React ecosystem. However, React comes with a massive ecosystem of component libraries, community support, and is used by a ton of major web applications. Not to mention, our team has tons of experience with it, and hiring more frontend developers in the future will be easy, since React is the de facto library everyone learns. While we most certainly could have used another library instead, development would have been slower, with far less benefit for us. Therefore, React was the best choice.

Choice 3 - State Management

• Redux
• MobX
• Remix
• ✓
  Redux Toolkit
• RxJS

This choice was quite a bit more difficult to make. As anyone who’s worked on even a slightly complex web app will tell you, state management is one of the most challenging things to get right. The old portal was built on top of vanilla redux, with a LOT of customization around it. It had become a giant tangled mess, as we added on websocket notifications, complex async interactions, “sagas”, and a whole lot more over the years. Probably one of the most enticing reasons to rebuild, from an engineering perspective, was to get rid of this complex, impossible state management and do things in a more sane way.

While personally I’m a big fan of the idea behind RxJS, it can be difficult to get started and is quite a paradigm shift if you’ve never used it before. For us, having prior experience with Redux meant a solution based on that would be ideal - we’re big fans of traceability and the abstraction Redux brings. There is a huge community around it as well and lots of tooling available.

One of the other important factors in our decision making was our API design. Cycle’s API is REST based, but has some live socket pieces to it as well that we need to utilize. Our state management system would need to gracefully handle local logic, API/asynchronous events, and streaming websockets. We had made the decision to codify our API into the OpenAPI spec to assist with downstream clients and documentation, so if we could find something that would tie into that for auto-generated API calls it would really tip the scales.

It just so happened that the perfect intersection of all these criteria was Redux Toolkit. Redux Toolkit is, as the name suggests, Redux based. However, it has several advantages that make it very appealing for our use case. First of all, it reduces and in some cases entirely eliminates boilerplate. There is an OpenAPI codegen for it that turns all of our API calls into React hooks automatically, while keeping everything type-safe. If the spec is updated, we’ll immediately know where we need to make updates in the portal in order to stay compliant.

What is really impressive about Redux Toolkit, however, is how it handles queries. There is a built in ‘tagging’ system, where every API call can have associated tags with it. For example, a call to get a container may have the response be tagged with {“type”: “Container”, “id”: abc123}. This is extremely useful, because at any point we can ‘invalidate’ the tag, and Redux Toolkit will automatically refetch that data and manage the cache for us in the background. Once you hook in Cycle’s notification websocket, it becomes trivial to automatically refetch or inline-update data based on the tags that would be affected by the notification.

The React components that manage data are ‘subscribed’ to these queries using the auto-generated hooks, so when the cache is changed or updated, the component re-renders. Redux Toolkit keeps track of ‘active’ subscriptions (is your component mounted?) and only updates things that actually matter. It’s a truly impressive feature and is the underpinning of the new portal, giving us flexibility, power, and accuracy all at once.

An honorary mention goes to Remix, and while we would have loved to use it for the portal, it was a bit limiting when it came to live data and state changes streaming over a websocket. We use Remix for everything except our portal (signup, status page, etc) but is better suited for website-like services rather than data-heavy dynamic apps.

The New Data Flow

With that context, it’s easy to see how the modern portal works to keep you informed.

1. The relevant components are rendered for the page you’re on, subscribing to the data they care about.
2. The cache for the subscription is empty, prompting it to make an API call to fetch the data.
3. The returned data is ‘tagged’ with the relevant tags.
4. A form is submitted implying something about our data was modified, or the notification socket sends us a notification that tells us our data is out of date (for example, someone else starts a container in the hub, and that container is tagged in our subscription).
5. The subscription cache is now ‘stale’ and since we still are subscribed to it, Redux Toolkit knows to make a background fetch for that data again.
6. The data is updated, and the component subscribed to the data is rerendered, showing us our container now ‘starting’.
7. We navigate away, and are unsubscribed from the data
8. After one minute, the cache is cleared out, so if we go back after that point, data will need to be re-fetched via API.

This is a simplified, but accurate description of the process in place today. There are some edge cases and slightly more complex situations we deal with (such as updating cache inline in some circumstances) but overall, the portal functions on these principles.

Choice 4 - UI and Component Library

• Material UI
• Ant
• Semantic UI
• Chakra
• Other Prebuilt UI
• ✓
  In-House
• ✓
  Tailwind + Headless UI

Once we made the decision to stick with React, the question became which UI/component library (if any) we wanted to use. While we were doing a redesign (see part 1 of this series), we ultimately wanted to keep many things the same, but improved.

With that in mind, we decided to utilize Tailwind for flexibility and ease of use, combined with some Headless UI components (all the logic, none of the style) and build out our own custom UI library in-house. Since there are many services around Cycle in addition to the portal, having a reusable set of components based on css classes seemed ideal. Overall, it proved extremely effective, and we are able to iterate on design very quickly.

Sharing Components Between Apps with Monorepos

To achieve reusability, we built all of our frontend applications into a monorepo powered by Yarn and TurboRepo. The combination of the two allowed for us to componentize our UI library for reuse across all of our frontend applications, without relying on an external package manager. We are able to sync and version things together, guaranteeing that everything at that particular git tag will work. We were able to easily share Tailwind configuration between apps, along with a core utilities library we made for shared functionality when working with Cycle API data structures.

Choice 5 - Testing

• Jest
• ✓
  Playwright
• Mocha
• Cypress
• React Testing Library
• Enzyme
• ✓
  Vitest

To round off our choices, the last major decision we made was around testing. Building an interface as complex as Cycle’s portal is one thing. Ensuring it continues to function as expected, without any regressions, is a whole different beast. With our new portal, came a new opportunity to improve on our admittedly imperfect testing of years prior.

The winners of Choice 5 were Vitest & Playwright. Vitest is nice because we’re already using Vite as our bundling tool, and Vitest slotted in nicely. Vitest runs all of our unit testing, and with Rust-like in-source testing, it’s very easy to construct tests right next to the functions they’re testing, without worrying about exports.

Finally, we decided to go with Playwright for integration and end-to-end testing. Playwright is a beefy test suite for ‘headless’ browser testing, giving us the ability to test full workflows in the portal, mocking requests from the API and making sure nothing breaks. We are able to codify edge cases and avoid regressions, giving us confidence when we deploy.

Tying it All Together

While these choices were far from all the technological decisions made over the development of the portal, they were by far the hardest and most impactful on the future of our engineering team. With this stack, we’ve not only been able to deliver a stable and performant interface for Cycle in under ten months, but have set ourselves up for a future where adding new functionality is a joy instead of a massive headache. In the next and final installment of this series, we’ll look toward the future of the portal, and what our users can expect to see in the coming months.

💡 Interested in trying the Cycle platform? Create your account today! Want to drop in and have a chat with the Cycle team? We'd love to have you join our public Cycle Slack community!