Scheduler Message Bus.

The environment scheduler supports a pub/sub style message bus for distributing messages between container in the environment, or outside the environment if the public setting is enabled on the scheduler.

Messages are sent to channels (optional) and tagged with a topic. Subscribers then open a stream which can be filtered by channel and topic alike. This entire system is part of the environment's scheduler service, which is also the service that manages function containers.

Main Concepts

The following concepts drive the main functionality of the message bus. First we have channels and topics which are the filters on the messages themselves and then some rules differentiating publishers and subscribers.

• Channels - are were a message goes. If its not defined the default channel is used as this is an optional filter.
• Topics - say what exactly a message is all about. This can be used by subscribers as a filter to be more granular about what they care about.
• Publishers - one topic and one channel, leaving the channel empty or not defining it results in the default channel being used.
• Subscribers - while each subscriber listens to one named channel (or default), subscribers can list multiple topics to listen for.

Subscribers must be connected / listening before the published message is sent.

Basic Example

A subscriber receives a message only when both of these are true:

1. It's on the same channel the message was published to, and
2. The message's topic is in its topics list, or it passed no list.

Fictional Acme Shipping Company Pub/Sub Setup:

Connecting to the Message Bus

The message bus can be reached internally by hitting the scheduler directly. The hostname for the scheduler is env-scheduler. For access over public networks, the scheduler must have the Public option in the config set to true and needs a LINKED record pointing to the scheduler container.

The endpoint /v1/message/bus is used for both public and internal connections and is used for both publishers and subscribers. Connections to the scheduler from public clients require using the scheduler access key. Not using the key will result in the following error:

{"error":{"status":403,"code":"403.not-allowed","title":"No valid access key specified"},"data":null}

We also offer a client library located here: https://github.com/cycleplatform/scheduler-api-client-ts which can be installed directly using:

npm i @cycleplatform/scheduler-api-client

While the client README covers usage, this client is generated directly from the Cycle OpenAPI spec. If so inclined, a user could generate their own client in their language of choice from that spec.

Publishing to the Message Bus

To publish, POST to the /v1/message/bus endpoint. Here is an example request:

curl -sS -X POST http://env-scheduler/v1/message/bus \
  -H 'Content-Type: application/json' \
  -d '{
    "distribution": { "channel": "shipping" },
    "message": {
      "topic": "orders.shipped",
      "annotations": { "source": "api" },
      "payload": { "order_id": 123, "carrier": "ups" }
    }
  }'

Each field is described here:

Subscribing to the Message Bus

To subscribe, run a GET to the /v1/message/bus endpoint. The example request is as follows:

curl -N -H 'Accept: text/event-stream' \
  "http://env-scheduler/v1/message/bus?channel=shipping&topics=orders.shipped"

The fields are described in the query parameters and are simply the channel and topics. If there are many topics they are a comma separated list.

When writing subscriber code, remember that the stream ends when a connection ends. This includes any scheduler restarts - so it may be necessary to pay attention to the connection state for long lived services.