Configuring Routes and Egress via Gateway.

Out of the box, Cycle offers a intuitive networking experience. For most users, advanced container and virtual machine networking isn't something they'll have to dive into. However, we generally expose advanced features through a building blocks type system.

For users that need to have granular control over their containers route tables or want to set a static egress IP for containers and virtual machines, this guide is for you.

The two features we will cover in this guide are:

1. Container Routing
2. Egress vis Gateway

Some Rules First

Routing tables exist in every container.

If you're not using specific routes or egress settings the route table is preconfigured to work by sending container to container traffic over eth-priv and traffic going out to the public internet over eth-pub.
Every container has both IPv4 and IPv6 route tables.

If there are connected SDN networks, there are additional entries for those interfaces that can be viewed on the IPv6 route tables.

When setting egress via gateway to true, the route table will be updated with a new default pointing at the gateway service.

Egress via gateway being a new "default" route is the reason why you can safely turn on both. Routes defined in the routes config will be more specific and not clash with egress via gateway.

Egress via Gateway

By default, all egress traffic originates from the server IP. In cloud environments, IP's can be dynamic as servers come and go and due to distributing container instances or vm's across multiple servers.

When Egress via Gateway is turned on, the downstream receiver of the traffic will see incoming requests as having originated from a static IP. This is accomplished by using the environments gateway service as the egress point. This leads to the origin IP being the same every time, regardless of which server the container or vm is on at the time of the request.

Enabling this Setting

In the portal, the setting can be enabled through the container or vm config page in the networking section. A more durable way to enforce this is through a stack, or even the API.

In the stack:

containers:
  worker:
    config:
      network:
        public: egress-only
        hostname: worker
        egress_via_gateway: true

Public, in the example, could be public, egress-only, or environment-limited.

Using the API:

{
  "action": "reconfigure",
  "contents": {
    "network": {
      "public": "egress-only",
      "hostname": "worker",
      "egress_via_gateway": true,
      "routes": null
    },
    ...: { /* the rest of the container config */ }
  }
}

Remember that when reconfiguring a container via the API the entire config needs to be set or default values will be assigned to the existing fields.

For users that plan to send a large volume of traffic egress over the gateway, there is an available performance setting on the gateway that can be toggled on. This setting will allow the gateway service to consume far more RAM and CPU from the server it is deployed to.

Container Routes

Container routes exposes a way for users to define additional entries into their containers routes table. For many users, this is an excellent way to route traffic over a VPN or some specialized network provided by a service like Tailscale.

Configuring Container Routes

Nexthops

Examples for Destinations

What actually goes in to the destination field? The ideal CIDR is as close to the range the private network uses as possible, not a bigger block that happens to have the private network range in it.

Defining in the Stack

Setting both of these in the stack is simple:

containers:
  worker:
    config:
      network:
        public: egress-only
        hostname: worker
        egress_via_gateway: true
        routes:
          - destination: 192.168.50.0/24
             interface: eth-demo
             gateway: 192.168.50.1

and the above results in the following:

Dive Deeper in Community

This guide is meant to be a primer for configuring advanced networking on Cycle using container routes and egress via gateway. If theres anything missing you'd like to dive deeper on, please open a new thread on the Cycle Community and a community member or a Cycle engineer will respond.

We love to hear more about what you're building, so don't hesitate to post.

You can also find us in our public Slack channel, if you're not already a member.