Showcase · Message bus · Go

messaging

One way to send a message between services: a controls-managed bus carrying CloudEvents, with the thing that carries them behind a swappable backend.

v0.5.1, released Pre-1.0Covers 8 projects

DocsRepository

The problem it exists for

A service that talks to another over a Go channel has to be rewritten the day the other end moves out of the process. An earlier attempt at a message bus here was designed directly against NATS, and it showed: a subscriber that panicked took the whole process down with it.

The deeper lesson was about abstraction. One with a single implementation is shaped entirely by that implementation, and an in-memory backend that copies NATS proves nothing about anything that isn’t NATS (spec 0001). So the contract was treated as provisional until something genuinely different arrived, and when SQS, AMQP 1.0 and RabbitMQ were specified, they forced nine changes to the core contract before any of them could be built.

How it works

Services talk to the bus, never the transport

A service imports the bus and CloudEvents, and nothing else, so moving from in-memory to NATS or a cloud queue is a choice of backend, not a rewrite. Under the five modules sit seven backends: in-memory, core NATS, JetStream, SQS FIFO, SQS Standard, RabbitMQ and AMQP 1.0. As well as publish and subscribe, a service can ask a question and wait for one answer, and every request is counted by how it ended. go/nats embeds a NATS server in the process, for tests and single-binary deployments.

Bounded, counted, and declared

Every subscription chooses how hard delivery tries. The default is at most once: one attempt, and a failure is counted rather than retried. At least once retries until the handler acknowledges, up to a limit, under a name the broker recognises after a restart. Asking for it on a backend that can’t survive a restart is refused unless the subscription says that’s fine. Every subscription has a size limit, and every message dropped at that limit is counted against the subscription it was meant for. Anything a backend can do beyond the basics it has to declare, and one shared conformance suite holds every backend to exactly what it declares.

Decisions and what they cost

  • The unit is a CloudEvent. Messages are CloudEvents, not raw bytes or any Go value. What it cost: anything that can’t be serialised can’t travel. Audio and video can, as bytes in an event’s data, but not efficiently. Each frame carries its own envelope and is copied once per subscriber, frames dropped at a full queue can’t be asked for again, and order holds only when one handler takes one message at a time (the limitations). A direct stream does that job better.
  • Bounded queues. Every queue is bounded and every drop counted, but what happens at a full queue is each backend’s to offer, rather than a queue bolted in front of every backend to make the options mean the same thing everywhere. What it cost: swapping backend can change what happens when a queue is full; core NATS can only refuse the newest message.
  • Generality has to be proven. Three backends unlike NATS were specified before NATS itself was built, with a shared conformance suite to keep them honest. What it cost: the nine contract changes, an SQS race suite that takes about eleven minutes, and a test image pinned to the last version that runs without an auth token.
  • Declare the edge case. JetStream declares that a server restart can use up a delivery attempt nobody saw (spec 0012), instead of hiding it or trying to rebuild the count from server advisories. What it cost: the count of exhausted messages can come up one short, measured at 11 runs in 250.

Proof in use

  • phpbotscout and comms-discord run on it, and so does Scout, its first user of request and reply.
  • The conformance suite runs five deliberately lying backends and requires every one to fail, after an audit found a lying backend could pass ten green cases. SQS Standard passes all 31 of its rows.
  • 22 releases across the bus and its four backends since September 2026, under controls for its lifecycle. A backend’s release merge request runs cicd’s go-core-currency check against the bus’s latest release.

Use it when, and when not to

Use it if your Go services need to talk asynchronously and you’d like to start in memory and move to NATS or a cloud queue later without rewriting them.

Nothing promises exactly once, so a handler subscribed at least once has to tolerate duplicates. A subscriber sees its messages in order only while it reads one at a time. Give it more readers and order goes, and nothing orders messages across subscribers. A message a handler turns down is redelivered straight away, with no backoff yet, so a handler waiting for a dependency to recover has to do its own waiting before it says no. Nothing routes traffic around a struggling consumer, and it isn’t a universal broker abstraction: anything past the two portable delivery shapes is a capability a backend declares. go/nats runs an embedded server in-process only, because it has no TLS or authentication for a listener yet, though a client connects to a cluster someone else runs just fine. go/schema, the schema registry, is at its first release and says itself it isn’t obviously needed yet.

Where it’s going

Backoff on redelivery, capabilities that depend on the connection, a deduplication window for embedded NATS, and keeping the message envelope swappable.

Backends, and the modules beside it

Last reviewed .