Showcases on PHP Boy Scout

How I work

Mon, 01 Jan 0001 00:00:00 +0000

The problem it exists for

Everything else on these pages was built by one person. There are 145 active projects in the estate, and AI agents do a great deal of the typing. That combination can go two ways. Either it’s a lot of plausible-looking code nobody really decided on, or it’s a lot of carefully decided code that happened to get typed quickly. The difference isn’t the agents, it’s the way of working around them, and that’s what this page is about.

It’s also the page I’d point a CTO at, because “how does one person keep 145 projects trustworthy?” is the question underneath every other showcase here.

How it works

Decide it in writing first

Nothing non-trivial gets built without a spec it implements. A spec records what done looks like, the decisions with what was rejected and what each one costs, and the questions still open, each one marked as either mine to answer or a fact an agent can go and establish. That split is what lets an agent work unattended overnight without quietly deciding things that were mine to decide. The specs live on each project’s wiki, and they’re public: go-tool-base alone has more than 200 of them. Every “decisions and what they cost” section on these showcases is lifted from them.

The reasoning stays findable

A decision is only useful if someone can find it later, so the specs are tied to the code and the docs from both ends. Code comments point at the decision that made them by number, and in go/errors’ words, “a change contradicting a D-number moves the spec first”. Across the Go modules, more than a thousand comment lines cite a spec that way. Where a decision depends on a fact nobody has, a short timeboxed experiment settles it first, and the result is kept as a dated report the spec cites. The Go module wikis alone hold 97 of them. Rejected specs stay up too, with their reasons, so the same idea isn’t argued from scratch a year later.

The docs carry the other half. Nearly every module’s documentation has a page saying what it deliberately doesn’t do, written as a decision rather than an apology: “Each of these is a decision with a name, not a gap waiting to be filled” (messaging’s limitations). Estate architectural patterns is what all of that keeps producing.

One session per project, and the tracker between them

Each project gets its own agent session, deliberately walled off so one can’t trample another’s work. When one turns up something another project needs, it doesn’t fix it in passing or try to remember it. It writes it up as an issue on that project, detailed enough that a session arriving with none of the context can act on it. Ready for human is how that came about, after I spent a while being the message bus myself and getting it wrong.

Every review is a claim, and a claim gets checked

An agent saying “fixed”, “tested” or “this doesn’t exist” is a claim, not a result. So a bug fix starts by proving the bug against the untouched code, with a failing test handed back before any change, because from the outside a real fix, a bug that never existed and a test written to pass all look the same (Prove the bug before you fix it). Reviews get the same treatment: a design worth arguing about goes to a panel of models from different vendors, told not to agree with each other for the sake of it, and what they find is evidence to weigh rather than a verdict to accept. How it’s tested covers what that looks like across the estate’s suites.

A human signs for every line

AI usage isn’t hidden anywhere here, and it isn’t credited in the commits either, because a person is accountable for every line that gets merged and an attribution trailer blurs who that is. Agents don’t cut releases. A release is a merge request that I merge, and on the infrastructure side even an apply waits on a human merging a plan that’s already been checked (Reviewed, then applied).

The habits, packaged

All of this lives as skills that any agent session can load, in a public plugin marketplace: 66 skills across 8 plugins at the last count, from spec-driven development and proving a bug first to timeboxing and running a review panel. Because a skill is effectively source code an agent runs, the marketplace has a security gate of its own on every change (the post on why).

Decisions and what they cost

Specs before code. Every non-trivial change has a written decision record first. What it cost: time up front on every feature, and a wiki per project to keep tidy, in exchange for agents that build what was decided instead of what sounded plausible.
Use, contribute, then build. I’d rather use somebody else’s tool, and try to fix it upstream when it nearly fits, before writing my own (Building it yourself is the third thing I try). What it cost: a lot of the estate is still my own anyway (colophon and go/chat are both third-rung builds), and each one is something I now maintain.
Isolation over convenience. Separate sessions per project, with the issue tracker as the only channel between them. What it cost: more issues than a team of one would normally write, and a handover has to be written as carefully as a spec.
No AI attribution in commits. Accountability stays with the person who merges. What it cost: the agents’ part shows up in the specs and reports, where it’s credited openly, rather than in the history.

Proof in use

116 projects release through colophon, where merging a release is a human action by design.
The showcases themselves: go-tool-base, chat, Signing and trust and colophon each cite the specs their decisions came from, and every one of those specs is public.
A terrible lead to an AI junior is the counterweight: the time the way of working failed, because I treated a problem as boilerplate and briefed it badly.

Borrow it when, and when not to

Borrow it if you’re one person, or a small team, trying to get a lot done with agents without losing track of who decided what. The skills install into Claude Code, most of them carry Codex instructions too, and the general ones don’t assume my projects.

Don’t borrow all of it at once. The spec discipline pays for itself on anything you’ll still be maintaining in six months and is overkill for a weekend script, and the panel reviews cost real quota.

Where it’s going

The newest additions are a timebox that reads the real clock instead of an agent’s sense of time, and a research panel that runs one question past several models and makes them argue it out. Both shipped in October 2026, and both came out of finding the gap the hard way first.

Estate architectural patterns

Mon, 01 Jan 0001 00:00:00 +0000

The problem it exists for

How I work is about the process: specs first, every claim checked, a human signing for every line. This page is about what that process keeps producing. How it’s tested is the evidence behind both. Across around a hundred Go modules and the tools built on them, the same design rules turn up again and again, and they’re there because each one was learned the hard way at least once.

If you’re deciding whether to depend on something here, these are the promises that hold across the whole estate rather than one module at a time. They fall into six groups: how modules are grouped into families, how each one is shaped, how it reports what happened, which way its defaults lean, what a tool is handed when it runs, and how the tools treat the people running them.

Module families, not module monoliths

The most obvious pattern in the estate is the family. Wherever several vendors do the same job (chat models, git forges, config stores, message brokers, key services), the work is split into a family rather than built as one module that knows about every vendor. A small core module holds the contract: the types, the behaviour every member has to have, and the tests that prove it. Each vendor, backend or file format then gets a member module of its own, which implements that contract and is the only place that vendor’s SDK is imported.

Eight families work this way, with 46 member modules between them. The biggest is config, with 26, and chat, git and forges, messaging and signing and trust each have a showcase of their own. A ninth, go/dispatch, is being designed now, in public: a core for submitting and supervising multi-node training jobs on the scheduler a team already runs, with Slurm and Kubeflow as its first members. Its contract is already written in the same shape as the rest, with support reported three ways per cluster and a failure’s cause never collapsed into “failed” (dispatch spec 0001). Splitting things up this way is what the next three patterns grow out of.

A program carries only what it uses

Most of the toolkit started life inside go-tool-base and was lifted out into modules of its own, so a project can take the one piece it wants without the framework around it (the extraction playbook, spec 0119). Families take that further. A program links only the members it imports, so a tool that only talks to Claude never carries the OpenAI or Gemini SDKs, and a tool that reads YAML never carries a cloud SDK.

The footprint is a design decision rather than a side effect, so it’s tested. Each module carries a test that lists everything the module pulls in and fails the build if a forbidden dependency turns up: the framework itself, its config and command-line libraries, telemetry or a cloud SDK. In the words of the controls module’s own test, a regression “fails here rather than silently bloating every downstream binary”.

It also makes what a binary can do a decision made in code, not in config. Each tool lists the members it links in one file, deleting a line takes that SDK out of the build, and nothing at runtime can put it back. In some families importing the member is all it takes to switch it on (chat, forge, comms, signing and encryption), and in others you construct it yourself (config and messaging). A linked plugin is never switched on by default either, because otherwise “an import list becomes a behavioural file” (go-tool-base). Moving the adapters out of go-tool-base’s own packages took a binary that uses none of them from 81 MB to 34 MB (spec 0194).

What it costs: a member that isn’t linked fails at runtime, and a missing import is the first thing to check.

One contract, proven on every member

Because every member implements the same contract, the core can ship one shared test suite that every member runs unchanged. The suite is itself tested against deliberately broken backends, and every one of them has to fail. The forge suite’s reason: “A guard that passes everything is worse than no guard, because it certifies nothing while looking like assurance.” Messaging runs five named lies, including a backend that claims to survive a restart and doesn’t (spec 0003). Every forge and messaging member runs one, as do the chat providers (spec 0011), comms, and config’s storage backends. A new vendor proves itself by passing the same suite as the ones already there.

The fakes behind each member’s unit tests are checked against the real service too, by a suite that runs in a container when asked for. A fake “encodes those same assumptions” as the code it stands in for, and “These tests are what stop that being circular” (config-vault). How it’s tested covers both kinds of suite, and the rest of the estate’s testing. What it costs: Docker in CI for the container suites, and the container library appears in fourteen repositories’ requirement lists, though never in a consumer’s binary.

A core release reaches every member

Members aren’t pinned to each other or to a matching version. Each one releases on its own schedule, for its own fixes and features, so chat’s providers and config’s adapters all sit at different versions. What the family does guarantee is that a change to the core reaches all of them. When a core releases, the bump lands on every member as a fix, usually opened by Renovate, so colophon cuts a new release of each one, and the whole family is out on the new core together. A check on every member’s release merge request flags one that would ship against an older core (the pipeline).

What it costs: every core release becomes a release of every member, so config’s 26 adapters each release again whenever the config core does.

The shape of a module

Settings in, no configuration reading

A toolkit module takes plain settings, as a struct or as options applied over safe defaults, and the application maps its own configuration onto them. Not one module outside the config family imports config. Messaging’s limitations page gives the reason: “A toolkit module that reads configuration decides for its consumer where configuration comes from” (typed config section adapters, spec 0115). Secrets arrive the same way, as a function the module calls when it needs one, never a stored string. Secrets in use covers that half.

What it costs: every application writes the mapping.

The standard library at every boundary

Logging, HTTP and the filesystem come in as the standard library’s own types: Go’s structured logger, its HTTP client, and one common filesystem interface. A module that logs nothing when you hand it nothing never writes to the global logger. The repo module’s docs give the history: an earlier logging dependency “pulled an entire terminal-UI stack transitively” (logging through the standard library, spec 0116). No toolkit module imports a third-party logging library. The tools built on go-tool-base use a terminal-friendly backend, but behind go-tool-base’s own logger, which hands the standard library’s handler to anything else that asks.

Depend on the narrowest piece

Modules expose small interfaces for each role, and a consumer depends on the smallest one that covers what it touches. go/repo has ten, from reading a tree to tagging. Adapters declare their own one-method slice of the vendor client they wrap, so a test can drive them with a fake instead of a cloud account. What it costs: adding a method to a published interface breaks every consumer’s hand-written fake. go/repo found none of seven consumers checked their mocks against it (spec 0002).

Errors that say what to do

The estate owns its error package, go/errors, with no dependencies at all. It replaced a library that cost 46 transitive packages for the ten functions in use (spec 0001). The convention it exists to carry is that the fix goes in the error, as a hint the user sees. A forge credential that’s configured the old way says so, and points at the migration table, rather than surfacing as a 401 far from the cause. 92 Go repositories require it, and none still carries the old library.

Creating an error and reporting it are kept apart. Deep code adds the context and the hint and returns the error. It never decides whether the program should stop. One place near the top does, with the exit code travelling on the error itself, so there’s “one exit path” in the whole program (the reporting model). Errors covers creating and reporting in more depth.

It’s one of several dependencies the estate took over once their upkeep had stalled, each with the evidence written down first. colophon replaced a release tool that “has had two human commits in its last sixty” (colophon spec 0001), and yamldoc became a YAML engine of its own (spec 0008).

Honest outcomes

“Don’t know” is an answer

Capabilities are three-valued: yes, no, or unknown. A provider that can’t say whether a model supports tools reports unknown, never no, because the two-valued alternative “is a lie which suppresses working features” (chat spec 0006). Only a confident no turns a feature off, and there’s deliberately no shortcut that squashes the three into a yes or no. The same shape is used for forge sites and comms channels. What it costs: every caller handles three cases.

Bounded by default, and loss is counted

Queues, reads, caches and waits all have a limit, and where something is left unbounded on purpose, the code says why. Going over a limit is refused rather than trimmed to fit, because “a clamp changes what the job means without saying so” (afmpeg spec 0044). Whatever is dropped at a full queue is counted against the subscription it happened to. Messaging’s reason: “An unbounded queue is not a policy. It is a decision to fail once memory is gone rather than at a number somebody chose” (full queues). Limits like these are in 25 Go modules, from messaging and NATS to the HTTP client, the rate limiter and every capped read.

What it costs: swapping a messaging backend can change what happens at a full queue, because each backend can only drop messages the ways it supports. And a call that can’t be cancelled, such as a keychain write, is bounded by giving up on it, so it may still finish after you’ve moved on.

Partial success says what it dropped

When part of a request can’t be honoured, you still get a working result, along with an error listing exactly what wasn’t applied. chat’s constructor “returns the best functional client it can build, alongside an error describing everything it could not apply” (chat spec 0007). The forge adapters do the same for a label that didn’t stick, and their error “never travels alone”, so finding it is a complete test for “this partly succeeded”. Messaging falls back to a backend’s own full-queue behaviour and says so, and the schema guard lets a message it can’t check through, but counts it (failing open). Anything the module is responsible for itself still fails outright.

What it costs: Go’s habit is to ignore the result whenever there’s an error, and a caller who does that gets the old all-or-nothing behaviour. The better one is there for callers who ask for it.

Silence is a failure

A green run has to mean the work happened. A skipped test, a tag whose release never appeared, a message delivered to nobody: each is turned into something visible, because “an absence does not show up in a log” (a skip is not a pass). When one project’s skipped tests started running, its coverage went from 52% to 91%. colophon warns about a tag with no release (spec 0018), and the message bus counts a publish nobody received.

What it costs: time and noise. That same project’s tests went from 1.8 seconds to 12.3, and making image tool bumps visible meant more tags and changelog entries across ten repositories (cicd spec 0081).

Never cache a failure

Clients for AWS, Azure, Google Cloud and Vault are built once and shared, through one helper that keeps a success and forgets a failure, so the next call tries again. Caching the failure would turn a passing network blip into an outage that lasts until the program restarts, which is why Go’s own build-once helper “is explicitly not an acceptable implementation of this contract anywhere in the estate” (never cache a failure). 17 modules use it. What it costs: while a provider is down, every call pays for another attempt.

Safe by default

Opt out of safety, never in

The constructor with no options is the hardened one. The HTTP client insists on TLS 1.2 or later, caps its connection pool and refuses a redirect from https to http, and “You opt out of these … never in” (hardened defaults). transit retries only requests that are safe to repeat, because “the safe direction has to be the quiet one” (safe defaults). forge sends a credential only to the host it was configured for.

The same goes for anything that reaches a person or leaves the machine. Telemetry, posting, mentions, uploads and AI providers are all off until asked for. afmpeg caps a job’s memory and time out of the box, because “A library whose safe configuration is opt-in gets deployed unsafely” (safe defaults), and krites sends no photograph anywhere unasked, since “A default that silently uploaded photographs would be a betrayal, not a convenience” (why providers are opt-in).

What it costs: the safe default sometimes gets in the way. The HTTP client speaks HTTP/1.1 only, and because its list of key-exchange curves is pinned, it doesn’t offer the hybrid post-quantum ones Go adds to its own defaults. An afmpeg job that needs more than 512 MB fails until someone raises the limit.

Refuse, rather than ignore or guess

A setting the module can’t honour safely is refused when it’s built or loaded, with an error that names it. It’s never dropped or guessed at. The embedded NATS server won’t listen on a network until it has TLS and authentication, because “the unsafe surface does not exist rather than existing with a warning next to it”. The NATS client refuses a URL with a password in it, and refuses to send a token over an unencrypted connection. Config refuses to write a value from a secrets backend into a plain file, and refuses an HCL file that uses variables or functions. Signing refuses a key format it would otherwise have misread. Code that parses input from outside is fuzzed for the property it protects, not just for crashes: one encryption target checks that bytes appended to a published certificate can never change the key a report gets encrypted to (go/encryption).

When a decision about trust or visibility has to be made without an answer, the restrictive branch wins. A forge that can’t tell whether a repository is public reports unknown, and callers treat unknown as not public, because there a wrong answer “is a disclosure, not a bug” (forge spec 0003). There’s one deliberate exception. Self-update goes ahead on the key built into the binary when the published copy can’t be reached, so a tool behind a broken proxy can still patch itself, though two copies that disagree always stop it. krites’ model downloads refuse in the same situation, because for them an unreachable key host “means try again later”. The stakes decide.

What it costs: capability withheld until it’s safe, which is a real gap for anyone who wanted it now.

Lifecycles that can’t be revived

Two rules from controls. A registered service is required, so one that’s unready makes the whole program unready, while a worker attached to a supervisor is watched but not required (spec 0002). And anything that can only be used once, like a listener or a server, is rebuilt from scratch on every restart, never revived. A stopped copy refuses to be used again (spec 0004). That rule came from five real failures across three modules, and controls ships a generic wrapper for rebuilding on restart, plus an analyzer that names the line where a service holds on to something it didn’t build.

Props: one typed context for what a tool needs

Every command in a tool built on go-tool-base is handed the same thing: Props, a single object holding what a command might need from its surroundings. The name isn’t short for “properties”. A prop is the beam that stops a structure falling down (Props).

In pattern terms, Props is a strictly typed encapsulated context object: one object carrying the services a unit of work needs, passed down instead of a long list of parameters. It’s built by hand in main, so it isn’t an IoC container. Nothing is registered, resolved or built for you, and what goes in is exactly what main put there.

It’s deliberately not Go’s own context.Context. A context carries values as untyped keys, so a command that wants the logger has to look it up, cast it and hope, and a missing one only shows up when that line runs. Props has a typed field for each service, so asking for the logger gets a logger, checked by the compiler, and every dependency a tool has is listed in one struct you can read. In go-tool-base’s own words, “Context is for cancellation and deadlines, not dependency wiring” (architectural decisions), so the two travel side by side and each does one job.

It’s a deliberate “god object”, and a function that takes the whole of it would hide which services it really uses, the way a service locator does. The narrow interfaces are what stop that: a function that only logs asks for something that can log, not for all of Props. rust-tool-base reaches the same outcome with a generic application type and strongly typed config instead (spec 0008).

A filesystem you can swap

Commands never touch the disk directly. They go through a filesystem on Props, which is the real disk in production and an in-memory one under test, and can be layered, for instance a read-only disk with changes held in memory on top. The test helper builds a complete Props with no real filesystem, network, keychain or process exit behind it, so tests that use it are safe to run in parallel.

Assets that merge

A tool ships its defaults, templates and docs inside the binary, so it works the moment it’s installed. Each command registers its own embedded files into the shared asset store, and opening a file reads across all of them. A plain file is taken from whichever command registered last. A structured file (YAML, JSON, TOML, CSV, .env) is merged from every copy, so a feature can add its own settings to the shared defaults without editing anyone else’s (asset management).

Logging and telemetry that are always safe to call

The logger and the log level are on Props, and --debug moves the level for everything at once. The telemetry collector is never empty: when telemetry is off, it’s a stand-in that does nothing, so a command records an event without first checking whether anyone is listening. What that telemetry may collect depends on whose data it is. Usage analytics is “the vendor learning about users and runs on informed consent”, so it’s off until the user opts in. Observability, meaning traces, metrics and logs for a service, is “the operator instrumenting their own service”, so configuring it is consent enough (observability).

The invocation’s own streams

Input, output and whether a person is at the keyboard come from Props too, not straight from the process. No package in the framework names the process’s own standard input or output, and a test enforces that. So a test drives a real setup wizard just by handing it streams, and a tool can redirect its terminal without the framework noticing.

What’s deliberately left out

Props carries no database. go-tool-base’s decision record turns one down because “Different tools need different data stores: imposing a database pattern would pull GTB into application framework territory” (architectural decisions). An event bus, a task queue and a caching layer were turned down for the same reason: the framework is a foundation for tools, not an application framework.

So each tool picks its own storage, and the estate’s default is the user’s own files. keryx keeps its work in files committed to git, and krites keeps its records in files beside the photographs. Where a tool really needs a database, it has the one that suits it: phpbotscout, a bot that runs all the time, uses SQLite, and Scout.DM uses Postgres.

What it costs: a container this broad is easy to lean on for everything, which is why the narrow interfaces exist. And a tool that needs a database wires its own, with nothing from the framework to start from.

Tools people run

One engine, several faces

keryx and krites each have a command line, a local web studio and an MCP surface for agents, and all three sit on the same code and the same files. keryx’s studio “maps 1:1 to a CLI operation on the same files” and “does not introduce a second source of truth” (keryx spec 0011). krites requires that its studio “exposes nothing the CLI/engine can’t do” (krites spec 0003). go-tool-base’s MCP mode runs every tool call as a command of the same binary, so an agent and a person get the same behaviour.

Scout.DM takes it a step further. Its faces are roles, named in config: the chat platform connection, voice capture, the machine-facing operator API, the operator console and the web portal. One process runs all of them by default, which suits a single container. A deployment can instead start them separately, as a set of services that each run some of the roles and reach each other over a NATS message bus. It’s the same binary either way, because “a role boundary that exists only in production is a role boundary nobody tests”. A role name it doesn’t recognise is refused at start, since a process that starts and runs nothing looks exactly like a healthy idle one.

What it costs: the studio is built into the binary by a generate step, so a plain go install of krites ships a placeholder in its place.

Irreversible verbs stay human

Agents get the commands that read, draft and generate. Anything that publishes, signs, spends or touches credentials is kept off the MCP surface “so an assistant cannot post publicly or rotate tokens” (keryx spec 0055). sigillum keeps signing back for the same reason: “a signature, once emitted, is a claim the project cannot retract” (sigillum spec 0003).

Where something runs unattended, it acts only on an approval a person has recorded in git. keryx’s scheduled posts send only what was approved, because “Approval is the point where a person takes responsibility” (keryx concepts). phpbotscout starts on every server in shadow mode, where “Everything runs; nothing posts” (phpbotscout spec 0010).

What it costs: a person stays in the loop for every one of those steps, and phpbotscout posts only after a person has read 30 of its answers and agreed they’re worth posting. Even then the software doesn’t switch itself on, because “nothing here lifts shadow mode” (phpbotscout). A person does.

Ask the world, and a re-run finishes the job

A tool’s own files are the default store (what’s deliberately left out covers why, and the exceptions). Where a tool needs to know what happened, it asks the thing itself rather than a record of it. colophon takes versions from git, “the only source that cannot disagree with reality”, after finding 9 of 231 recent tags in the Go group weren’t on their default branch (what actually landed). skillup is built on the same idea. It takes a plugin’s previous version from the history of the plugin’s own manifest rather than from the last tag, because “A marker is a second source of truth, and a second source of truth is something that can disagree with the first” (why not tags).

That makes a re-run safe. colophon “finishes the release it started instead of adding a second one beside it, and says which of the two happened” (colophon spec 0014).

What it costs: a read before every write. And keryx keeps large media in object storage rather than git, which handles it badly.

A preview that writes nothing

Deciding, writing and pushing are separate commands, and the one that decides writes nothing and needs no credentials. colophon’s plan “Writes nothing: no branch, no merge request, no tag. Needs no credentials, so it is safe to run against any checkout” (colophon). skillup’s plan changes nothing and its apply writes one file without committing, keryx has a dry run on its writing commands, and config plans a change before applying it. What it costs: more commands to learn.

The irreversible step goes last

Every step that can fail runs before the one that can’t be undone. rust-tool-base’s updater downloads, checks the signature and checksum, unpacks and runs the new binary’s self-test before swapping it in, so “the disk holds either the old version or the fully verified new one”. colophon creates the release last, because a release with nothing on it “answers yes to the only question a consumer knows how to ask”, and two live breakages were recorded before that was fixed (complete releases).

And once released, nothing changes: “A broken release is corrected by cutting a new tag” (colophon spec 0025). What it costs: a mistake stays published, so the fix is always another release.

Decisions and what they cost

Own the error package. Replacing a one-maintainer dependency meant migrating every module, and the saving only lands when the last one moves. The spec says so: a partly migrated estate carries both.
Families, not a monolith. The extraction playbook’s own verdict is that moving the tests is the bulk of the work, and a family of modules means a release round across every member whenever its core changes, which a monolith never needed.
Generality, proven first. Messaging specified three backends unlike NATS before building NATS, and they forced nine changes to the contract (messaging). That was slower, and it’s why the abstraction holds.
The stakes set the default. Fail closed where a wrong answer leaks something or runs untrusted code, and fail open only where refusing would strand a user, as self-update does. What it cost: the estate is deliberately inconsistent here, and each exception has to be written down.

Proof in use

98 Go repositories, 54 of them in eight families, and every one is pre-1.0. A breaking change ships as a minor version, with the migration written into the release notes before it merges (colophon).
go/errors is required by 92 of them, and 34 of the toolkit modules take the standard logger.
The tool patterns hold across keryx, krites, colophon, sigillum, skillup and go-tool-base, and Rust’s framework reaches the same outcomes with its own idioms.
The patterns are written down for agents as much as for people. The estate’s Go module skills teach them to every session that creates or uses a module (Agent skills).

Borrow it when, and when not to

Borrow these if you’re building a library other people will import, and want them to take only what they need. The dependency-budget test and the lying-backend suite carry over to any language with a dependency lister and a test runner. The tool patterns carry over to any command-line tool that acts on someone’s behalf, and most of all to one an agent will drive.

They cost more up front than a single package does: more modules, more release coordination, and a mapping layer in every application. For a single application that will never be imported, most of this is overhead.

How it's tested

Mon, 01 Jan 0001 00:00:00 +0000

The problem it exists for

When agents write a lot of the code, a green test suite is easy to come by and hard to believe. A test written to pass passes, a suite that skips everything is green, and a check that has never failed may not check anything. So the estate’s testing is built around one question: has anyone seen this fail? In the words of the estate’s own test-first skill, “A test you never saw fail has not been shown to test anything.”

How I work covers the process and Estate architectural patterns the design. This page is the evidence behind both.

How it works

Red before green

A change starts with a failing test, and a bug fix starts by proving the bug against the untouched code. The estate’s skills hold every agent session to that: write the test, “watch it fail”, then write only enough code to pass it, and when something breaks, “You do not get to theorise until you have a command that goes red on this bug” (test-first-discipline). Prove the bug before you fix it is the story of why.

Every Go repository under the race detector

97 Go repositories run their tests through one cicd component, with Go’s race detector on, a coverage report and a summary of what was skipped (go-test). Its time limit is deliberately twice Go’s default, because a race-enabled suite near the limit “fails with a goroutine dump that reads exactly like a deadlock and is not one”. go-tool-base bans the package-level mocking hooks that would race under parallel tests.

Suites that prove themselves

Every family’s core ships one test suite that each member runs unchanged: 14 repositories run config’s backend suite, and the chat providers, forge adapters and messaging backends each run theirs. A suite like that is only worth something if it can fail, so messaging, forge and comms test the suite itself against deliberately broken backends. Messaging’s reason: “A capability that lies is worse than one that is absent”. Each broken backend runs in a child process, and the test “passes here only if it failed there” (messaging). chat and config check theirs against a simple in-memory provider that’s correct by construction, and none of the suites brings in an assertion library, so a provider that runs one takes on no new dependency.

A skip is not a pass

Tests that need a real service, a network or a built binary are always compiled and skipped unless an environment variable asks for them, never hidden behind a build tag, where they “rot silently” (env-gated tests). 37 repositories gate suites this way, and 14 of them start the real service in a container. A skip still has to tell the truth. When the variable is set and the dependency is missing, the test fails, because “Somebody asked for them, and they cannot run” (a skip is not a pass). That rule came from a green pipeline whose skipped tests, once running, took one package’s coverage from 52% to 91%.

Fuzzing for properties

73 fuzz targets across 12 repositories, most of them named for the property they protect rather than for “doesn’t crash”: a decryption that never emits unauthenticated text, a routing table that can never bypass discovery, a render that always keeps the body. The seeds replay in every ordinary test run. Four repositories also run a short fuzz search on every merge request, because four review rounds, “roughly 11M tokens of agent time”, never found the defect that one did: “A fuzz target found it in three seconds” (go/encryption). In yamldoc, which has 40 of the targets, “Fuzzing is a merge gate, not a nightly job”.

Properties over snapshots

A snapshot test compares output byte for byte with a saved copy, and the estate mostly avoids them. ffmpeg-wasi checks properties of its output instead, such as stream counts and durations, because a snapshot suite “goes red on every FFmpeg bump, which would make the instrument useless at the exact moment it is needed” (ffmpeg-wasi). Where snapshots are kept, they’re pinned to a reference on purpose: keryx’s card renderer is compared with images from the renderer it replaced, and regenerating them from the new one “would make the test agree with whatever the code now does, which is the same as deleting it”. sigillum turned a defect that came back six times into one structural test, because “Written as six separate findings they look like six unrelated bugs. Written as a property they are one.”

Mutation as a ratchet

Mutation testing changes the code on purpose and checks that a test notices. sigillum runs it on five packages on every merge request and fails if a change goes unnoticed. It calls it “a ratchet, not a detector”: it was added while hunting a class of defect and didn’t find it, because “The line was covered; the state was not” (sigillum). comms-discord does it by hand. A row of its contract suite only counts as covered once its check has been watched failing under the change the row names.

Scenarios for whole workflows

Behaviour that spans a whole command or a lifecycle is written as Gherkin scenarios and run against a built binary with godog. go-tool-base uses them “strategically for CLI workflows and state machine scenarios”, with table-driven unit tests as the baseline (architectural decisions), and in go-tool-base a new command needs scenarios before it merges. Ten repositories have them: go-tool-base has 290, keryx 121, colophon 97, Scout.DM 65 and krites 55. The Rust crates use the cucumber equivalent.

What a unit test can’t see

A dependency budget. 86 repositories carry a test that lists everything the module pulls in and fails the build when something forbidden turns up (go/redact).
Reproducible images. Seven build images are built twice from scratch on every change, and the job fails if they differ, “rather than asserting that it is” (go-tools).
Docs against the code. krites’ docs are checked on every merge request for commands that don’t exist, by a test that has itself been watched failing (krites).
The right architecture. go/encryption and sigillum run their suites as 32-bit builds too, because one overflow test asserts nothing on a 64-bit machine, and “A test that only means something on one architecture has to be RUN on that architecture, or it is decoration.”

Front ends and Rust

The web front ends are tested with Vitest, testing-library and svelte-check, and the studios with Playwright. Before that runs, a hard check confirms the browsers are actually installed, because the earlier job “failed in 12 seconds and allow_failure: true reported that as a green pipeline for three runs” (svelte-test). All 12 Rust repositories run cargo-nextest, because some tests “only pass with nextest’s process-per-test isolation”, alongside clippy with warnings as errors, cargo-deny and a coverage floor that fails the build (rust-test).

Decisions and what they cost

Runtime gates, not build tags. What it cost: container suites need a privileged runner, and a run with the variable off reports a lot of skips.
The race detector on every run. What it cost: a slower suite, and a real hang takes the full 20 minutes to be reported.
Fuzz on merge requests. What it cost: the budget is seconds per target, so a deep search still happens on a developer’s machine, and in sigillum “a quiet week fuzzes less than a busy one”.
Mutation where coverage lied. What it cost: twelve seconds a run in sigillum, and the reminder it carries: “Do not mistake a green run for evidence the suite is adequate.”
Two coverage rules. Rust’s floor fails the build. go-tool-base checks every package against 90% but only reports, so it “can never flake the pipeline red”. What it cost: a Go package can slip below the line until someone reads the report.

Proof in use

97 Go repositories under the race detector, and 12 Rust repositories under nextest.
73 fuzz targets in 12 repositories, four with a search on every merge request.
732 Gherkin scenarios across ten repositories.
86 repositories with a dependency budget, and seven images checked for reproducibility on every change.

Borrow it when, and when not to

Borrow it if agents write a lot of your code, or if your tests have ever been green while something was broken. The lying-backend suite, the skip that fails when it was asked to run, and the short fuzz search on merge requests carry over to any language with a test runner.

It costs pipeline time and a privileged runner for the container suites, and the habit of watching every new test fail first is slower than writing it to pass. For a script you’ll throw away, most of it is overhead.

Agent skills

Mon, 01 Jan 0001 00:00:00 +0000

The problem it exists for

An agent reads its instruction files and does what they say, so those files behave like code. Copied from one repository to the next, they drift. Accept them from strangers, and anyone can hide an instruction inside one.

So the habits behind How I work live in one place, a public plugin marketplace: 66 skills across 8 plugins, plus 5 subagents and 21 commands, installable into Claude Code or Codex. Versioning is where it bit first: before it was automated, one plugin shipped nothing to anyone for six weeks, because its version number never moved.

How it works

One folder per plugin, one catalogue

Each plugin is a folder with an owner and a version of its own, and a single catalogue lists them all. You add the marketplace once and install the plugins you want.

A security gate on every change

Every merge request runs a shared security check over the skills. It fails a change on hidden characters (zero-width or Trojan Source tricks), on an invalid manifest or skill header, and on leaked secrets, and it warns on instruction patterns that look like an injection. Two more checks make sure a skill is declared the same way for Claude and for Codex, and that the prose doesn’t carry the usual AI tells. The scanners are cicd’s skill-security component, so any repository of skills or agent instruction files can include the same gate, and the pipeline keeps them current. The marketplace I had to defend from my own attack surface is why.

skillup: versions that can’t stall

A plugin’s version number is what delivers its update. Claude Code caches each plugin under its version and never looks at a git branch or tag, so in the words of its own docs, users only receive updates when that field changes. A version that doesn’t move doesn’t delay a change. It withholds it, with no error and nothing in CI to notice.

skillup is the tool that keeps every version moving. It treats each plugin in the marketplace as a segment with its own version, and it has four jobs:

plan shows what each segment’s next version should be, and changes nothing.
apply writes those versions into the manifests and does nothing else: no commit, no tag, no push. The contributor commits the bump alongside the change that earned it, so a plugin’s content and its version never disagree on the main branch.
check is the gate. It fails a merge request when any segment has fallen behind, and it also reports a plugin the catalogue doesn’t list (so nobody can install it) or a catalogue version that contradicts the plugin’s own.
tag adds a tag per segment at the commit that set each version, after the fact if need be.

The trick is where it starts counting from. Release tools usually take the last git tag as the baseline, which makes tags something a person has to keep cutting on time, forever. skillup takes each segment’s baseline from the history of its own manifest, the commit where its version last changed. So it’s correct on a repository that has never been tagged, a late tag costs nothing but visibility, and tags can be added afterwards because the history still says where every version was set.

Its rules are the cautious ones. A change counts towards a segment by the files it touches, not by the label on the commit. It never lowers a version, because commit messages under-describe some changes (removing a skill breaks its users, but tends to land as a refactor), and reaching 1.0 is the author’s promise to make, not a sum. Below 1.0 a breaking change is a minor bump. It edits the version and nothing else in the file, rather than rewriting the JSON and turning a one-character change into an eighteen-line diff. And running it twice before committing doesn’t bump twice.

It’s a go-tool-base tool, and all its git work goes through go/repo, so there’s no git binary to shell out to (git and forges). It runs as a gate on GitLab or GitHub, with a setup guide for each, and Your first run with skillup walks through one.

docscheck

docscheck is a smaller sibling, checking that the commands a tool’s docs show you are commands the tool can actually run. krites runs it on every merge request through cicd’s docs-verify component, which runs a project’s own docs check with no change detection at all, because a docs gate that only wakes up for docs changes misses the code changes that break the docs.

Decisions and what they cost

Version from history, not tags. skillup reads each plugin’s manifest history instead of the last git tag. The existing tools were tried first: one had no monorepo support, one was GitHub-only, one bumped every file to a single version, and the closest fit took its baseline from tags. The tag-based tools proposed 0.1.0 for every plugin on an untagged repository, and gave two different states of the main branch the same version (the spike). What it cost: about 600 lines of Go to maintain, repeating parsing other tools already do.
CI checks, people bump. The pipeline only checks the version; the contributor runs the bump and commits it. A pipeline that pushed its own bump would start no new pipeline, leaving the merge request waiting forever. What it cost: a manual step on every change.
Deep members decide. The research panel puts one question to two “deep” agents, who give the verdict, and one “broad” agent, who only steers, each working on a throwaway copy of the files inside a sandbox (spec 0002). What it cost: one of the three tools has no read-only mode, and on macOS the sandbox can’t keep a member’s own state apart.
A clock the agent can’t ignore. The timebox skill comes with hooks that read the real clock and stop an agent quitting before its time (spec 0001), because a skill alone relies on remembering to look. What it cost: Codex gets the skill without the hooks.

Proof in use

It’s what this estate’s own agent sessions run on. The marketplace has had 292 commits since June 2026, and its plugins version on their own: the common plugin, the biggest, is at 0.46.
skillup gates every merge request to the marketplace, pinned at v0.3.1.
docscheck checks krites’ docs: 57 files and 97 documented commands, every real error caught and no false alarms.

Use it when, and when not to

Borrow it if you work with AI coding agents and want your habits written down once, installed everywhere, and safe to accept changes to. The general skills don’t assume my projects.

skillup only understands Claude Code plugin marketplaces, writes no changelogs, and needs the full git history (a shallow clone gives an answer that’s quietly wrong). docscheck checks command paths only, not flags or prose. The review panel needs a second agent tool with quota to spare, and falls back to one agent, visibly, without it. Hooks don’t carry over to Codex, and output styles are Claude-only.

Where it’s going

Rust and Terraform skills, a checker for skills that have drifted from their own repositories, and the panel and timebox findings deferred from their first release.

AWS account foundation

Mon, 01 Jan 0001 00:00:00 +0000

The problem it exists for

Every new AWS account needs the same four things before anything useful can happen in it: somewhere to keep infrastructure state, a way for CI to sign in without stored keys, guardrails, and alerts for when something goes wrong. Working those out again by hand for every account is, in the words of the module’s own spec, waste. So they’re two modules, applied in order, and the second is built without leaning on anybody’s framework: 52 AWS resources across six parts, and no third-party modules among them.

How it works

First, the bootstrap

The bootstrap module does three things and deliberately stops there. It makes an encrypted state bucket that refuses to be destroyed (with locking inside S3 itself, so no DynamoDB table), a role CI can assume through OIDC from GitHub or GitLab with no stored credentials anywhere, and a generated configuration for aws-nuke, so a throwaway account can be wiped clean on purpose.

Then the baseline

The security baseline is applied through that CI role, and comes in six parts, each of which can be switched off:

account hardening
an audit trail in CloudTrail, kept for two years
a configuration history in AWS Config
threat detection, with GuardDuty, Security Hub running the AWS best-practice and CIS standards, and Access Analyzer
alerts for high or critical findings, and for any use of the root login
an operator role that needs MFA and is locked to one region

Decisions and what they cost

Keep the bootstrap narrow. State, CI sign-in and the nuke configuration only, instead of one module that sets up the whole account. What it cost: two modules and two version streams to keep in step.
No frameworks. Cloud Posse, Gruntwork and Control Tower were all considered and turned down, and even the community modules the first draft planned to use for three of the baseline’s parts ended up written by hand. What it cost: every line is mine to maintain, and getting the audit trail’s bucket protected the way I wanted took about fifty more of them.
Half rewritten. GitLab’s OIDC sign-in was written by hand, but GitHub’s still wraps the community module, because rewriting both would have pushed existing users through a state migration. What it cost: two code paths for one job until 1.0 tidies it up.
No lock table. State locking uses S3’s own locking instead of DynamoDB. What it cost: it needs OpenTofu, or Terraform 1.10 or later.

Proof in use

The bootstrap is at v0.3 and the baseline at v0.2, both from July 2026, and both docs sites are live. Each release tag publishes the module to GitLab’s module registry through cicd’s tofu-module-publish component, and cicd’s guide to signing its plan and apply jobs in to AWS can reuse the bootstrap’s CI role.
Why I hand-rolled every module is the argument behind building them this way.

Use it when, and when not to

Use it if you’re standing up a single AWS account and want the boring, important parts done properly before you build anything in it.

Know what it isn’t. It’s one account at a time: AWS Organizations, Identity Center, WAF, Inspector, Macie and Detective are all out of scope. GuardDuty runs in the main region only. The CI role gets administrator access unless you narrow it, which you should. Security Hub and GuardDuty cost money, and a brand-new account can refuse them until it’s subscribed, which is why each has its own switch. And it’s below 1.0, so pin a tag.

Where it’s going

The region lock the baseline gives its operator role is meant to reach the bootstrap’s CI role too, and separate plan and apply roles with a permission boundary are on the roadmap. Neither has been built yet.

Build images

Mon, 01 Jan 0001 00:00:00 +0000

The problem it exists for

Every CI job runs inside a container image, and the easy thing is one big image with every tool in it. The estate had two. The development image was 977 MB to pull and 2.7 GB unpacked, carrying 2,582 scanner findings. So a job that only needed curl and jq to post a Discord message pulled 2.36 GB to do it, and a Rust repository inherited 66 serious findings, none of which came from Rust (spec 0074).

How it works

One image per kind of job

Everything starts from ci-base. It’s built on Wolfi, a minimal Linux distribution made for containers, pinned to an exact image digest, and carries little more than bash, git, curl, jq, Python and sigillum, so any job can sign what it builds. On top of that sit one image per track:

Image	For	Carries
go-tools	Go lint, tests, security and releases	Go, golangci-lint, goreleaser, govulncheck, syft, and the ONNX Runtime library
rust-tools	Rust lint, tests, docs and security	rustup with clippy and rustfmt, cargo-nextest, llvm-cov, deny, audit, semver-checks, release-plz
node-tools	Svelte builds and tests	Node and npm
playwright-tools	browser tests	node-tools plus a headless Chromium
tofu-tools	OpenTofu modules	tofu, tflint with the AWS rules baked in, terraform-docs
docs-tools	the docs sites	zensical
release-tools	releases	colophon

A job pulls the image for its track and nothing else. Most run as an ordinary user rather than root. The Go and Rust images run as root because the runner checks code out as root.

Built the same way twice

The images are built with Buildah, with timestamps fixed to the commit’s own time and the build history left out. Then a check builds the image a second time from scratch and fails if the two don’t come out identical, printing exactly what differed. Getting there meant tracking down seven kinds of litter tools leave behind (caches, telemetry files, compiled Python timestamps, stray files in /tmp) rather than blaming the builder (spec 0098).

Scanned by who can fix it

Each image is scanned twice, split by who can act on the result (spec 0099). Findings in the operating system packages fail the build, because a rebuild clears them. Findings inside upstream tools are reported but don’t fail it, because only that tool’s maintainer can fix them. docs-tools fails on both, because its Python packages re-resolve on every build, so a fix lands without anyone acting. Every image is rebuilt and rescanned every night without publishing, so a vulnerability disclosed after a release doesn’t sit unseen until the next one.

Kept current by bots that must release

Every tool version is pinned in the image’s build file with a note telling Renovate where to look for new versions, so adding a pin is all the wiring it needs (spec 0065). Patch and minor bumps merge themselves on a green pipeline after a three-day wait. Major bumps wait for a person. A bump to anything an image ships is committed as a fix, so colophon actually cuts a release for it, because an image that changed without a new version once shipped an old colophon for a day (spec 0081). The new image then reaches the component library’s defaults the same way.

Every download checked

Each tool is fetched with the checksum its upstream publishes and refused if it doesn’t match. Go is checked against go.dev’s own release index. Rust tools come in prebuilt only, through cargo-binstall, because only one of the six publishes a checksum to check against. ONNX Runtime comes from the estate’s own signed artefact channel.

One deliberate exception

ffmpeg-wasi-build is the pinned toolchain behind ffmpeg-wasi’s builds: compilers, the WebAssembly SDK and build tools baked in once, rather than installed by ten build jobs from a Linux mirror. It exists because one day in September that mirror slowed to 50 kB/s, handed back a package index with the wrong checksum and failed the pipeline. It stays on Debian, not Wolfi, because the native driver it builds promises users a minimum C library version (it runs on Ubuntu 22.04, Debian 12 and RHEL 9), and that’s a contract.

Decisions and what they cost

Wolfi. Docker’s hardened images are built to have no shell or package manager, which a CI job needs, and Alpine’s C library can’t run several prebuilt tools (spec 0074). What it cost: the same package set is 81 MB bigger on Wolfi than on Debian, and the spec says not to sell it on size. The Debian layer alone carried 2,468 findings against Wolfi’s none. Its packages can’t be pinned to versions, because nothing tracks Wolfi’s, so a nightly freshness check stands in.
One per track. What it cost: a fix to ci-base takes three release hops to reach a job, and nine repositories’ build setups to keep in line.
No version manager. One was measured on tofu-tools at 23 seconds against 9, with two update bots instead of one and no checksum checks. What it cost: every tool’s checksums arrive in a different shape, handled one by one.
Playwright in an image of its own (spec 0076). Baking it into node-tools would take that image from about 117 MB to about 780 MB for a job two projects use.
Reproducible over fast. What it cost: builds 14% to 67% slower, and a merge request costs about three builds. Builds match from one run to the next, not across weeks, because Wolfi’s package index keeps moving.
Retag, don’t rebuild, on release (spec 0094). Pushing, not building, was most of a release pipeline: 354 of 597 seconds on go-tools. A release now retags the image main already built and tested. Only release-tools does so far.

Proof in use

119 of 143 estate repositories use at least one of these images, mostly through the component library: release-tools in 105, go-tools in 102, docs-tools in 72 and rust-tools in 12.
55 releases across the nine images in about eight weeks.
go-tools against the old development image: 484 MB against 958 MB to pull, and 15 serious findings against 73. rust-tools scans clean on its own.
Compressed sizes today run from 72 MB for docs-tools and 78 MB for ci-base up to 525 MB for go-tools and 830 MB for rust-tools.

Use it when, and when not to

They’re public, so anyone can pull them, but they’re built for the estate’s own jobs and components rather than as general-purpose images, and they’re x86-64 only.

The images themselves aren’t signed and carry no software bill of materials or build provenance, and colophon’s own signature isn’t checked in release-tools, only its checksum. A checksum catches a corrupted download, not a substituted one. ffmpeg-wasi-build lags the rest: it’s still built with kaniko, without the reproducibility check or the nightly scan. The component library’s defaults also run a release or two behind the newest images, and 13 repositories still pin the retired development image directly.

Where it’s going

Retagging on release for the other eight images, with ci-base last (cicd#70), moving the Rust release component onto rust-tools (spec 0104, cicd#71), signed images with provenance (cicd#72), bringing ffmpeg-wasi-build in line with the rest (ffmpeg-wasi-build#1), and moving the last 13 repositories off the retired image (cicd#73).

chat

Mon, 01 Jan 0001 00:00:00 +0000

The problem it exists for

Talking to one AI provider from Go is easy. Talking to several is where it goes wrong: every vendor SDK has its own idea of a conversation, a tool call, a stream and an error, and pulling three of them into a project drags in three dependency trees whether you use them or not. The usual answer is a big framework, and for AI integration the de facto standard is LangChain (in my opinion, anyway). It’s capable, and it exists for a real need, but it’s a lot to swallow when what you want is “send this, get an answer back”.

go/chat is the client behind gtb ai, pulled out so a project can use it without go-tool-base and without linking every vendor’s SDK. How heavy an unused dependency gets came up early: the MCP SDK alone was 1.35 MB and eight modules for a consumer that never touched it (spec 0021), and making it opt-in (spec 0028) is the same thinking applied to everything else.

How it works

One conversation, whichever provider

You get one small client: add a message, chat, ask for a structured answer straight into a Go value, hand it some tools, and read back the history and the usage. Streaming, persistence and caching are there when a provider can do them, and absent rather than faked when it can’t. Around that sit the things every real use ends up needing anyway: a tool loop, fallback to another provider when one falls over, history that stays within bounds and compacts itself, and encrypted persistence.

Opt in to the providers you use

Every vendor lives in a module of its own, switched on with a single import, and the core depends on almost nothing (a test fails the build if a vendor SDK, a CLI framework or OpenTelemetry ever sneaks into its graph). Ten providers across five modules today, and a sixth on the way:

Module	Providers
chat-anthropic	Claude through the API, and Claude Code running locally
chat-openai	OpenAI, any OpenAI-compatible server, and Codex running locally
chat-gemini	Gemini, Gemini on Vertex, and Antigravity running locally
chat-bedrock	Amazon Bedrock
chat-openai-azure	Azure OpenAI
chat-vllm	A self-hosted vLLM server (in progress, not released yet)

The local ones are the interesting bit for me: they drive a coding agent’s own CLI, so a tool uses whatever that CLI is already signed in with, instead of needing an API key of its own. chat-mcptools bridges your tools into those local agents over a loopback MCP server, so they can call them too.

What a model can do is data, and “don’t know” is an answer

Each provider reports what it supports (streaming, tools, structured output, caching and the rest) and every answer is one of three values: yes, no, or unknown. Unknown is the default, and it means “go ahead and find out”, because the alternative is worse. As the code puts it, reporting a provider that can’t be asked as unsupported “is a lie which suppresses working features”. Only a confident no changes behaviour.

The answers come from the caller first, then a live probe, then a catalogue generated from models.dev and refreshed on a schedule. The catalogue is built ahead of time and never fetched while your program runs.

Every provider sits the same exam

A conformance suite ships with the core, and every provider module runs it to prove it behaves like the others: capabilities, structured answers, history, timeouts, streams, errors, concurrency and refusals, nine groups of them. It’s how ten providers from five vendors end up feeling like one client.

Decisions and what they cost

Not LangChain. Before a line of it was written, LangChain Go, go-openai, Vercel’s AI SDK and about ten others were weighed up, and none of them fitted: the aim was something much lighter and simpler, an interface small enough to fit on one screen (the design notes, and the post that argued it). What it cost: owning it, and it has grown a long way since. It started as four methods and five providers inside go-tool-base, and it’s now its own module family with ten providers, a tool loop, fallback, history compaction and capability catalogues. It’s still a long way short of what LangChain does, which is rather the point, but “small” has had to stretch.
Three-valued capabilities. A fixed vocabulary of capabilities, each yes, no or unknown, instead of a bool (spec 0006), because almost nothing a caller needs to know is really a bool. What it cost: Claude Code locally and an arbitrary OpenAI-compatible server can only ever say unknown, and unknown can’t warn you about anything.
Degrade, don’t refuse. A misconfigured setting doesn’t stop you getting a client: you get one that works, plus a list of the settings it had to drop (spec 0007). Failing hard, dropping them quietly into a log, and a strict-mode switch were all turned down. What it cost: that list lives in the error you get back, and nowhere on the client afterwards. The post on it has the whole argument.
Conformance ships in the core. The suite lives inside go/chat itself instead of a separate module (spec 0011), because a fourth thing to version would have been one more thing to drift. What it cost: the dependency test had to be narrowed to just the packages that reach a consumer.
Structured output. Structured answers use each provider’s own structured-output feature instead of a forced tool call wrapped in instructions (spec 0024). The spike behind it had 15 of 15 models accept it, and 29 of 29 replies came back valid against the schema. What it cost: about 35 billable calls to find out, and a change to how those answers sit in the history.

Proof in use

Five projects in the estate use it: go-tool-base for its AI commands, keryx to write social copy, krites as the critic that reviews a photo cull, phpbotscout to answer support questions, and Scout.
39 releases of the core since 12 July 2026, and between them the provider modules have shipped another 120. Each provider’s release merge request runs cicd’s go-core-currency check, so a provider can’t ship against a core that’s moved on unnoticed.
The conformance suite runs in six suites across five provider modules, and the family carries 836 test, fuzz and example functions.

Use it when, and when not to

Use it if you want an AI provider (or several, with fallback between them) in a Go program without inheriting every vendor’s SDK, and you’d like structured answers, tools and streaming to look the same whichever one you’re talking to. Install a provider module and let it pick the core version for you.

Don’t use it for anything beyond chat. There’s no image generation, embeddings or audio, no rate limiting or spend cap, and one client talks to one provider at a time (fallback is for when one falls over, not a router). Claude Code locally can’t stream, and a refusal is only spotted where the provider marks it as one. The limitations page has the full list. And it’s pre-1.0, so the provider modules move with the core’s minor version.

There’s a Rust one too, if that’s your language: rtb-chat, part of rust-tool-base.

Where it’s going

vLLM is next. Spec 0035, approved on 10 October 2026, adds chat-vllm for a self-hosted vLLM server, as a module of its own with its own HTTP client rather than a layer over the OpenAI provider. It ships two things the other providers will get later, once a live run has confirmed them: token log-probabilities and tool calls handed back without running them. It streams, but every capability that depends on the model answers unknown, which is the right answer for a server that can run whatever model you load into it. Its first release waits on that live run against a GPU server.

colophon

Mon, 01 Jan 0001 00:00:00 +0000

The problem it exists for

When a release merge request merges, something has to decide which commit gets the tag. The usual candidates are the merge commit, the squash commit or the head the merge request recorded, and under GitLab’s automatic rebase that recorded head can be a commit that never made it onto the branch at all. Nothing fails when that happens… the tag just quietly points somewhere the branch never went.

So I went and counted. Across my own Go modules on 1 August 2026, 9 of 231 recent tags weren’t on their default branch.

How it works

Everything comes from the commits

colophon reads commits and commit messages, and nothing else. It never takes an answer from a merge request, an issue or anything else the forge holds. The version is worked out by a pure function over what has landed since the last tag (no clock, no network, no forge), the changelog is written from the same commits, and when a release merges, the tag goes on the commit found on the target branch itself rather than whatever the merge request says it merged.

That’s why plan works from anywhere. Point it at any checkout, on any forge or none, with no credentials at all, and it tells you what version your commits have earned and which commits decided it. It’s also the easiest way to compare colophon against whatever a project releases with today, because it writes nothing.

The forge is where colophon writes: the release branch and its merge request, the tag, the release, and the odd comment. It just never reads its answers back from there.

The release, in four commands

Command	What it does
`plan`	Works out the next version and the commits that earned it. Writes nothing and needs no credentials.
`propose`	Re-cuts the release branch from its target, writes the changelog and opens (or updates) the release merge request.
`publish`	Once that request merges, resolves the commit that actually landed on the target branch, tags it and creates the release. Safe to re-run.
`release`	Creates the release for a tag that already exists, which is the way back when colophon can’t release itself.

The version a release carries comes from the changelog at the tagged commit, never from the merge request’s description, so a tag and its version can’t drift apart. The docs explain what “actually landed” means in full.

Trailers: saying what a commit means

Because the commits are the only input, anything you want to tell colophon goes in a commit message, mostly as a trailer in the last paragraph:

Annotation	What it does
`feat:`, `fix:`, `type!:`	The Conventional Commits subject decides the bump and the changelog entry. `perf` and `refactor` release a patch too.
`BREAKING CHANGE:`	Marks the change breaking. Below 1.0 that’s held to a minor bump, because declaring a stable API should be a decision rather than something a footer does to you.
`Release-As:`	Sets the version outright, which is how a project promotes itself to `1.0.0`.
`Release-Note:`	Adds a line of prose to the release notes (a fenced `release-note` block takes anything longer).
`Updates:`	Tells an issue, a merge request or a wiki page what became of the work once the commit lands, such as `Updates: #13 shipped` or a spec page moving to `IMPLEMENTED`.

(A bare Closes #13 at the bottom doesn’t knock out the trailers above it, which it would for git interpret-trailers. colophon reads a little more than git does there, on purpose.)

After the merge: apply and announce

Two more verbs run once the work has landed.

apply runs on every push to the target branch and carries out the Updates: trailers that push brought in, commenting on an issue or merge request, or setting a wiki page’s status. It never decides for itself that something is finished. A record changes because a commit said so, and each comment carries a key naming the commit, so a retried pipeline posts nothing twice.

announce tells people a release happened, through whichever adapters the project’s .colophon.yaml opts into. There are two today. discord posts the release to a channel, and the headline projects in the estate announce on the community Discord that way (a Slack adapter is coming soon). file appends the release to a YAML or JSON file in another repository and commits it. The file adapter is how this site’s changelog gets fed, and it’s how the version at the top of this page stays current. Announcing is the one thing colophon does that isn’t idempotent: a webhook can’t be read back, so a retried Discord announcement goes out twice, and the docs say so rather than pretending otherwise.

In a pipeline

Nobody wires those verbs up by hand. cicd’s colophon component turns them into pipeline jobs, and 118 projects include it. It also adds the parts that only make sense in a pipeline:

publish runs ahead of propose, as the decision below needs.
A check on the release merge request fails it if it carries anything but the changelog files colophon writes, because under a fast-forward merge any other commit on that branch would end up tagged.
A project with binaries gets its release on the tag pipeline instead, carrying links to what cicd’s goreleaser component built.
After each Go tag, the Go module proxy is asked about it once, so pkg.go.dev and the family checks in the pipeline see the release within a minute.
Announcements default to this blog’s release feed, and a failed announcement can be re-sent with one button.

Decisions and what they cost

Build, not fork. releaser-pleaser was releasing 96 of my projects when I started, it had had two human commits in its last sixty, and a fix I’d written for exactly this bug sat upstream unacknowledged. A fork would have meant maintaining two things instead of one. What it cost: I own a release tool now, and the library it stands on is no small thing (go/forge alone is roughly four times the size of the tool it replaced). The reasoning is in spec 0001.
Sequencing over parity. Parity with releaser-pleaser was never the goal. I own every project that uses it, so where releaser-pleaser’s behaviour was just its answer, colophon was free to pick a different one. What it cost: a release tool’s bugs become permanent tags, so the rollout went cicd first, then the forge family, then everything else, and never as a flag day.
Forge-independent. It was designed from the start to talk to forges only through go/forge, so GitLab, GitHub, Gitea, Codeberg and Bitbucket all work through their adapters, and a forge it has never heard of can be added as somebody else’s module. What it cost: only GitLab is proven by daily use (it’s what this estate runs on), and I’d rather say that up front than imply otherwise.
Publish first. Publish runs before propose, as separate jobs. Between a release merging and its tag existing, colophon would happily work out the same release all over again, so publish has to close that window first. They’re separate jobs because propose only sees the new tag if its checkout is fetched after publish pushed it. What it cost: a pipeline rule that’s easy to break by folding the two together, so colophon also refuses to propose a release whose commit is already at the tip.
Exit 0 means it worked. apply used to report a failed instruction and still exit zero, and a job went green on a 404 because of it. The first fix on the table was an opt-in flag so nobody’s pipeline would go red on upgrade… and I dropped it for a plain non-zero exit, leaving the colour of the pipeline to allow_failure. What it cost: upgrading turned some green pipelines red, on purpose. The whole story is here.

Proof in use

116 of the estate’s 145 active projects release through it (measured on 10 October 2026), and nothing runs releaser-pleaser any more. The rest are the Rust crates, which still use release-plz, and things that never cut a version at all, like the sites and this blog.
colophon cuts its own releases, so every tag on its repository is one it resolved itself.
The CLI’s contract is held by a feature suite that runs the real command code against a mocked forge (spec 0020).
Releases across the estate announce themselves to this site through the file adapter, which is where the changelog and the latest-releases list on the home page come from.
The headline projects announce each release on the community Discord through the discord adapter.

Use it when, and when not to

Use it if your project uses Conventional Commits, releases through a merge or pull request, and you want the tag on the commit that actually merged, whichever forge you’re on. It isn’t tied to a language either: all it asks for is the commit convention and a CHANGELOG.md it maintains for you. On anything other than GitLab, try plan first. It writes nothing, and support there comes from the go/forge adapters rather than from me using it every day. And don’t use it yet if you need a stable CLI. It’s pre-1.0, so pin the version.

Where it’s going

Rust is next. The Rust crates still release through release-plz, and the decision to move them to colophon was made on 5 October (spec 0027). Colophon’s half has landed, and the publishing to crates.io and the semver check that go with it are cicd’s to build (cicd#69).

config

Mon, 01 Jan 0001 00:00:00 +0000

The problem it exists for

Most configuration libraries merge every source into one big map: files, environment variables, flags, defaults. That works right up until you need to know where a value came from, because the merge throws that away. And it really goes wrong when the library writes config back by saving that merged map. Comments vanish, the order goes, defaults the user never wrote get pinned into the file, and a password that came from an environment variable lands on disk. Viper’s old write path did exactly that, and it’s the reason go/config exists.

The editing half is measurable. Across eight real config files, editing one key at a time for 70 edits in all, yamldoc changed exactly one line every time and lost no comments. The usual YAML libraries got there 24 times out of 70.

How it works

One store, and it remembers

go/config reads every source into one store, in the order you add them, and records which source supplied each value as it goes, so you can ask it to explain where a setting came from. Reads come from a frozen snapshot, so a group of related reads never mixes values from before and after a reload, and a reload that fails is rejected while the last good config stays live.

Writes go back where they came from

A change is written to the file that already owns that key, and never to an environment variable or a flag. yamldoc does the editing: it changes only the bytes of the edit, then re-reads its own output and checks it before anything is saved.

A module per format and per backend

Only YAML is built in. Everything else is an adapter of its own, so you compile only what you use: seven formats (JSON, TOML and HCL read and write; XML, dotenv, INI and Java properties read only), four filesystems, three object stores (S3, Google Cloud Storage and Azure Blob), and eleven key-value and secrets backends, from Consul, etcd and Vault to the cloud secret managers and the macOS keychain. An adapter is handed a client you’ve already built and never holds credentials of its own.

Decisions and what they cost

One owner for config. A single store owns all config input and output (spec 0002), instead of bolting a writer onto the old container, which would have needed locks and fingerprints only because three components shared the files. Replacing Viper had been estimated at 8,000 to 12,000 lines of work; once the store existed, the code left to replace was 781. What it cost: a breaking release, and real porting at a dozen call sites in go-tool-base. The writer I added and the Viper I deleted tells it.
Its own filesystem interface. A six-method interface config defines for itself, over keeping afero (23% of the dependency graph on its own), the standard library’s read-only one, or one that can’t be swapped out in tests (spec 0003). What it cost: afero users wrap it themselves, or add the adapter.
A module per format. One module for each format and each remote system, never a bundle (spec 0004, spec 0005), so nobody compiles every parser and cloud SDK to read one file. What it cost: twenty-seven modules to release, and every config core release becomes a release of all of them. A format per module, not a fatter config core is the argument.
A YAML engine of its own. yamldoc moved off the parser it was built on to an engine it owns (yamldoc spec 0008). What it cost: an edit is four to seven times slower than the plain YAML libraries (about 4 ms against 0.6 ms on a 12 KB file), and by its own docs’ account, it’s young. YAML has comments for a reason is the launch.

Proof in use

go-tool-base, colophon, keryx, krites and phpbotscout all read their config through it, and go-tool-base uses twenty of the adapters.
385 releases across the 28 modules since July 2026, with the core at v0.20. Twenty-seven adapters releasing against one core is easy to get out of step, so every adapter’s release merge request runs cicd’s go-core-currency check, which flags a release still built against an older core.
yamldoc edits real config files one line at a time, measured above, in colophon and keryx as well as config itself.

Use it when, and when not to

Use it if your tool layers config from several places, needs to say where a value came from, writes changes back, or reloads while it runs.

Don’t, if you have one file, a few environment variables, no write-back and no reload: the README says plainly that Viper serves that well. It also has limits worth knowing: there are no transactions across sources, writes aren’t coordinated between processes, validation understands only required fields and enums, and a key with a dot in it can’t be reached. Explaining a value prints the value, secrets included, so mind where that output goes. yamldoc is strict, so a sloppy file fails, and its edits are slower on big files.

Where it’s going

Loading a Kubernetes ConfigMap or Secret mount as one layer per file is next, along with yamldoc’s release qualification and a few more editing operations for comments, tags and anchors.

controls

Mon, 01 Jan 0001 00:00:00 +0000

The problem it exists for

A long-running Go program usually runs several things at once: web servers, gRPC servers, background workers. They all have to start, report whether they’re healthy, restart when they fall over, and shut down in a sensible order without hanging. Some of them are there for the life of the program, and some come and go while it runs: a worker per tenant, a subscription per topic. Most teams write that supervisor again for every project, and then write the glue that plugs each kind of server into it again too.

How it works

The controller: the services a program needs

controls has two layers. The controller looks after a fixed set of services, registered before it starts. It starts them all at once, gathers their health checks into status, liveness and readiness reports, restarts a failed service with growing waits up to a limit, and stops them in reverse order of registration within a time limit. A stop that hangs is abandoned at the deadline and named in a warning, so a shutdown always finishes. It depends on nothing but go/errors.

The supervisor: workers that come and go

The supervisor looks after children that attach and detach while the program runs, under the same restart rules, and stops them all at once. A supervisor is itself a service, so it registers with the controller like anything else: the controller supervises the supervisor, and the supervisor supervises the children. A program has one controller and as many supervisors as it has changing sets.

What separates the two layers is what a failure means (spec 0002). A registered service is required, so one that’s unready takes the whole program out of rotation. An attached child isn’t. When it fails for good, the supervisor stays ready, reports degraded, and hands the failure to the code that attached the child, which knows whether it mattered.

Common services, registered in one call

The services most programs run already know how to plug in, so wiring them up is a call rather than a page of start and stop code:

transport registers an HTTP server, a gRPC server or a REST gateway with a controller in one call, which brings its start, stop and status with it. It also turns the controller’s health reports into ready-made health, liveness and readiness endpoints, and into the standard gRPC health service.
messaging’s bus has the start, stop and readiness methods a controller expects, and it uses both layers: the bus is a service of the controller, and each subscription is a child of a supervisor the bus owns. A restart replaces that whole set of subscriptions at once. The bus also offers a readiness check that reports degraded, by name, when a subscription has failed for good.
go/nats registers an embedded NATS server, and a client connected to it, with their start, stop, readiness and liveness wired up.

Decisions and what they cost

One owner for signals. The supervisor stopped catching Ctrl-C by default: a standalone program asks for it, and the old opt-out was deleted outright (spec 0001), instead of every framework tool having to remember to switch it off. What it cost: a break for anyone relying on the old default, and a survey found nobody was. It fixed a live bug on the way.
Registered means required. A failed child never makes its supervisor unready, at any proportion, including all of them (spec 0002). Otherwise one dead worker would reach the whole program’s health through the supervisor’s own registration. What it cost: a child that really was needed has to be noticed by the code that attached it, which then makes itself unready.
Generations. Five recorded failures across the estate came from a restart reusing something that could only be used once. Now stopping always finishes within its limit, and an old and a new copy never run at the same time (spec 0004). What it cost: leftover work from a stopped copy may still be running, so it’s cut off and refused any further access. controls ships an analyzer that names the line where a service holds on to something a restart can’t reuse, and cicd’s go-singleuse component runs it on merge requests as an advisory check. No project includes it yet.
Say why it stopped. A controller now records why each shutdown happened and who failed on the way, and its failure events never hold a service up (spec 0008). What it cost: a reader that stops taking events can delay shutdown by the rest of the time limit. It’s on the main branch, not yet in a release.

Proof in use

Ten public projects run under controls, including go-tool-base, messaging, keryx, krites, phpbotscout and the transport stack.
17 releases of controls since July 2026, and over 200 of its own test functions.

Use it when, and when not to

Use it if a Go program runs several long-lived services, or workers that come and go, and you want them started, watched, restarted and stopped properly, without a framework around them.

It serves no health endpoints itself (transport turns its reports into them), doesn’t order startup or make one service wait for another, and a crash in a service’s start code takes the program down. A supervisor can’t be reconfigured while it runs, restart waits double with no randomness, and there are no metrics or tracing. An HTTP or gRPC server registered through transport can’t be given a restart policy yet, because a Go server can’t serve again once it has been shut down.

Where it’s going

Letting a controller stop when a service gives up, surviving a crash in a service’s start, finishing cleanly when the work is done, and one event queue for the controller and its supervisor.

Docs at estate scale

Mon, 01 Jan 0001 00:00:00 +0000

The problem it exists for

There are 146 active projects in the estate, and every one of them has docs that can go stale without anyone touching them. The case that made it obvious: three of krites’ pages told readers to run a command that had stopped working when it gained subcommands. The merge request that fixed those pages ran only the security jobs and the docs build, because a docs-only change skips the code gates, so nothing would have caught the next one either (spec 0068).

The readers changed too. By September the docs sites were 71% of the estate’s search impressions, AI crawlers were fetching hundreds of pages a week, and assistants were fetching keryx’s Instagram how-to on users’ behalf when Google sent it nobody (spec 0089). So the docs have to be right, findable, and readable by a machine that will quote them.

How it works

One site per project, built by one component

73 public projects have their own docs site, at <name>.phpboyscout.uk or <module>.go.phpboyscout.uk, built with Zensical by cicd’s zensical-pages component. Each one builds on every docs change and deploys from main. The 41 adapter modules (each config backend, chat provider and forge adapter) deliberately have no site of their own and are documented on their family’s. The toolchain lives in a build image rather than each repository, after hand-maintained lock files left 18 update merge requests stuck at once (spec 0039).

Written for the machines too

Because every site is built by the same component, an improvement lands everywhere without a change in any repository. The component writes an llms.txt map of each site for AI agents (spec 0089), adds structured data naming the site, the software and its author on every page (spec 0091), honours the exclusions Zensical ignores, so scaffold pages stop being indexed (spec 0090), and fills in the “edit this page” link, which had pointed every site at a branch that doesn’t exist (spec 0092).

The same shape everywhere

Every site follows Diátaxis: tutorials to learn from, how-to guides for a task, reference to look things up, and explanation for the why. 62 of the 73 sites have all four. How-to and reference pages open with the answer, in under a hundred words before the first example. Tutorials live in each project’s own docs, and when the blog runs one, it’s a copy that points back to the docs as the original. A tool built on go-tool-base gets the layout generated for it. The rule itself is a skill every agent session reads (diataxis-docs).

Specs don’t live in the docs. They’re decision records, so they go on each project’s wiki, and the docs link to them. The test for what moves is whether a page would need updating when the code changes.

Checked against the code

docs-verify runs a project’s own docs check on every merge request, with no filter on which files changed, because the code change that breaks a page doesn’t touch the page. The component supplies the guarantee and the project supplies the check. docscheck is one such check: it reads the command examples in a tool’s docs and fails when one names a subcommand that doesn’t exist. It’s narrow on purpose. The obvious broader rule “found 15 problems on a corpus whose real defect count was 3.” Rust projects get the same from cargo doc treating warnings as errors, on all 12.

Beyond the gates, two agents in the estate’s plugin marketplace check docs on demand: one audits every signature, flag, config key and snippet against the source and fixes nothing, and the other writes to the Diátaxis shape and finishes with a merge request.

Answerable by the bot

phpbotscout answers support questions from these sites, with citations, so it doubles as a test of whether they answer anything. In August it reviewed the docs of 48 projects. Across 829 pages it found reference was the weakest part and mattered most, that the “why” had been left in specs the bot can’t read, that limitations weren’t written down, and that sections didn’t stand alone when retrieved one heading at a time. Every one of those reviews was closed the next day. keryx gained a reference section, a limitations page and its first tutorial. config gained six reference pages documenting all 21 of its errors.

Decisions and what they cost

One map writer. The alternative was “66 merge requests to add a file every site derives identically”, and a hand-written map drifts (spec 0089). What it cost: the generator lives inside the component’s template, with a self-test that its copy matches the canonical one. A full-text dump for language models was turned down as “a training-data gift with no evidence anyone fetches it”.
Builds follow docs changes. A site only rebuilds when its own docs change. What it cost: a component improvement reaches a site only on that site’s next docs change. 21 module sites still have neither the map nor the structured data.
A narrow checker. What it cost: docscheck checks command paths only, so a renamed flag or a wrong argument still gets through.
Tutorials moved into the docs. What it cost: every blog tutorial was copied back and rewritten in the tutorial register.

Proof in use

73 docs sites, all live, with 1,652 pages between them. The largest are go-tool-base’s (268), keryx’s (128), and config’s and cicd’s (72 each).
61 sites have at least one tutorial, and 38 have a limitations page.
52 sites already carry the AI map, and 50 the structured data.
48 bot-led reviews, then 72 follow-up passes comparing each README and its docs with each other and with the code, 44 of them finished.

Borrow it when, and when not to

Borrow it if you run more than a handful of projects and want their docs consistent without editing each one: put the build in one shared component and every site gets each improvement. Diátaxis and an answer-first rule carry over to any docs tool.

Only krites has a docs check wired into docs-verify so far, and docscheck checks command paths only. No site has automatic link checking: links are checked by the auditing agent, and after a restructure. Zensical has no redirects of its own, so a renamed page needs a redirects file. Two sites, cicd’s and phpbotscout’s, name their GitLab repository as their own address, so search engines and the map point there rather than at the site. go.phpboyscout.uk is hand-written and has no map.

Where it’s going

Rebuilding the 21 sites that predate the map and structured data (cicd#75), pointing cicd’s and phpbotscout’s sites at their own addresses (cicd#76, phpbotscout#22), and the 28 follow-up passes still open, such as keryx#25 and config#7.

errors

Mon, 01 Jan 0001 00:00:00 +0000

The problem it exists for

The estate used to sit on cockroachdb/errors, chosen for its features and for being well maintained. The features held up and the maintenance didn’t: a bug that hid user hints underneath a joined error went unanswered for months, and a docs typo sat open for two years while a fix CockroachDB itself needed merged in five days (Merged in five days, open in seven hundred tells that story).

And it was expensive for what it gave. The estate called about ten of its functions, and for those paid 46 of go/forge’s 376 transitive packages, a deprecated protobuf library among them (spec 0001).

How it works

The same names, nothing underneath

go/errors keeps the names and signatures the estate was already using, so moving across was mostly a change of import path. It imports the standard library and nothing else, and a test fails the build if that ever changes. On top of that it adds what was missing: hints a user can act on, structured attributes that carry straight into logs, sentinel errors that don’t capture a stack, and a join that behaves the way the standard library’s does.

A handler that doesn’t exit for you

go/errorhandling sits on top for command-line tools: it reports an error to the user with its hints, carries an exit code on the error itself, and hands that code back to main instead of exiting the process from inside a library.

Decisions and what they cost

Standard library only. The core depends on nothing else, not even “a few” dependencies. What it cost: a narrower API (attributes reuse the standard logger’s type), and an errors package the estate owns outright.
Source-compatible. The names stayed the same rather than being redesigned to taste. The spike that proved it moved go/forge across with one sed over 19 files and no code changes. What it cost: keeping some names I wouldn’t have picked.
A fixed join. Join behaves properly instead of copying the old bug. What it cost: a behaviour change at four call sites across the estate.
Return the code, don’t exit. The handler reports and returns an exit code; it never calls os.Exit itself (errorhandling spec 0002). What it cost: every main has to act on the code it’s given, and go-tool-base lost a workaround it had needed to flush output before exiting.

Proof in use

92 projects in the estate require go/errors, from go-tool-base and colophon to 83 go/* modules, and nine use errorhandling.
cockroachdb/errors no longer appears in a single go.mod on the main branch, directly or indirectly.

Use it when, and when not to

Use it if you want errors with hints and structured attributes, compatible with the API you probably already know, and you’d like your error package to be the one dependency that brings no others.

It doesn’t send errors over the wire yet, so matching an error across a process boundary doesn’t work, and there’s no OpenTelemetry integration. Stacks are capped at 32 frames, a sentinel has no stack unless you wrap it, and the handler doesn’t redact or translate anything. The limitations pages for errors and errorhandling have the rest.

Where it’s going

Rendering an error onto a trace span is the next piece, and after that a wire codec, so an error can cross a process boundary and still be recognised on the other side.

Git and forges

Mon, 01 Jan 0001 00:00:00 +0000

The problem it exists for

A program that works with version control talks to two different things. One is the forge, the website and API around the code: its releases, merge requests, issues and wikis. The other is git itself: cloning, committing, branching and tagging. These modules cover both, and neither needs a forge’s SDK or a git binary installed.

go/forge has a narrow job: letting a program work with a forge properly, the same way whichever forge it is. It started inside go-tool-base as a thin layer over releases, finding a tool’s latest release and downloading its assets so the tool could update itself. Once it was its own module it grew quickly, because programs want far more from a forge than its releases: merge requests, merging, issues, comments, wikis and the rest.

The hard part is that no two forges do the same thing the same way. GitHub, GitLab, Gitea and Bitbucket each have their own idea of what a release, a merge request or a label is, and Bitbucket simply doesn’t have some of them at all. go/forge is one consistent interface over all of that, so a tool written against it doesn’t need four code paths, or four vendors’ SDKs in its build.

There’s a less obvious reason too. A forge’s own answers can’t always be trusted. On GitLab, the rebase that happens just before a fast-forward merge is never written back to the merge request, so the head commit the merge request records can be one that sits on no branch at all. A release tool that believes it tags the wrong commit, quietly. Measured across one group of my own modules on 1 August 2026, 9 of 231 tags weren’t on their default branch (spec 0010).

On the git side, go/repo starts from the fact that git doesn’t know what a forge is. It wants a URL and a credential, so cloning from GitHub shouldn’t put a GitHub SDK in your build either. It sits on go-git, git written in pure Go, which is powerful and raw. Its biggest catch is concurrency: go-git isn’t safe to use from more than one goroutine at once, even for reads, so a server or a tool doing several things in parallel can corrupt go-git’s caches just by reading. go/repo’s headline job is making go-git safe to share.

How it works

A small core, and everything else optional

Every forge has to do four things: find the latest release, find one by tag, list releases, and download a release asset. Everything beyond that (merge requests, merging, publishing releases, issues, comments, wikis, pages and more, nineteen of them) is an optional capability, each one separately present or absent, and a caller checks before it asks. Adding a new capability never breaks an existing provider, including somebody else’s, and the capability matrix says never where a platform simply can’t do something and not yet where the code hasn’t been written.

One module per forge

Each forge’s SDK lives in an adapter of its own (GitHub, GitLab, Gitea and Codeberg, Bitbucket), plus a plain “direct” provider for download URLs that needs no SDK at all. The core can’t take on any of them: two tests fail the build if a forge SDK, or even an instrumented HTTP stack, reaches its dependency graph. A shared conformance suite holds every adapter to the same contract, and a provider registers itself by name, so a forge I’ve never heard of can be added as another person’s module.

Careful with what it trusts

A token is pinned to the host it was issued for, so a hostile release can’t redirect it somewhere else. And “which commit actually merged” is only ever answered after checking that the target branch contains it, never from the merge request’s say-so.

go-git updates its internal caches even when it’s only reading, so two goroutines reading the same repository can corrupt those caches. A read-write lock looks like the obvious fix and isn’t one, because the reads write too. go/repo’s thread-safe repository puts every operation behind one exclusive lock instead, reads included, and the rest of the design makes that lock hard to slip past:

A filesystem handle taken from it keeps taking the lock after it’s been passed somewhere else, and so does every file opened through it, so code that knows nothing about the lock still can’t touch the working tree unsafely. There’s no way to reach the unlocked filesystem underneath.
A sequence that has to happen as one, such as writing two files and committing them, runs inside a callback that holds the lock for the whole sequence.
Reaching past the wrapper to go-git itself also happens inside a callback, under the same lock, so even the operations go/repo doesn’t wrap stay safe.

It’s safe mostly, and the edges are deliberate. The lock isn’t reentrant, so calling the repository from inside one of its own callbacks deadlocks. A go-git pointer kept after its callback returns, or one handed out at setup time, is back outside the lock. And the lock makes sharing safe rather than faster: operations queue up behind each other.

A real repository, held in memory

go/repo can clone a repository into memory and work on it there: read files, commit, branch, tag and push, with nothing written to disk. A tag is always annotated, is made at a full commit SHA the caller has already settled on rather than whatever a name resolves to, and refuses to overwrite a tag that already exists. It isn’t a mock. It’s a real git repository, so it also makes a test fixture with no fake behaviour to drift from real git, and nothing to clean up afterwards. The same API works against a checkout on disk.

A sparse checkout clones a large repository but fills the working tree with only the directories you name, in one option. On a test repository of 300 files it cut the memory a clone held from 12.6 MiB to 5.4 MiB. The history is still fetched in full, so the saving is the working copy.

The working tree is an ordinary filesystem

The checked-out files are exposed as an afero filesystem, the interface much of the Go ecosystem already reads and writes through, by way of aferobilly. It’s a live view rather than a copy, so a file written through it is a file in the working tree, with no sync step.

Signing in to any forge without importing one

Cloning or pushing over HTTPS means sending a username and a token, and each forge expects a different username beside the token: oauth2 on GitLab, x-token-auth on Bitbucket, and x-access-token on GitHub, Gitea and Codeberg. So a git library has to know which forge it’s talking to. The obvious way to know is to import a forge library, and go/repo used to do exactly that, pulling in a whole release-and-forge package to get two values the caller already had: a name and a token.

Now the caller passes them in. The forge is a plain name, so an unknown one, a self-hosted or brand-new forge, simply gets the GitHub convention and works without a code change. A test fails the build if any forge package or SDK reaches go/repo’s dependencies, so the coupling can’t creep back.

The token is fetched only if it’s actually used. SSH takes priority over a token, and reading a token can mean unlocking the system keychain, so a repository set up for SSH must never trigger that prompt. The tests prove it with a token source that crashes if it’s ever called.

And a changelog

Beside both sits go/changelog, which builds a changelog from a repository’s Conventional Commits, and can read an existing set of release notes back into a structure a program can query.

Decisions and what they cost

Every forge, or none. No capability ships for one forge alone. GitHub’s adapter came out of go-tool-base carrying merge requests, repository creation and file contents, and only its releases came across, because a GitHub-only method would compile, pass its tests and still make the portability claim false (spec 0001). A capability lands when all four adapters can do it, or a platform genuinely can’t and says so. What it cost: that GitHub code waited until the other three caught up.
No recorded head at all. The contract has no field for the commit a merge request claims to have merged. The answer comes back only once the target branch is confirmed to contain it (spec 0010), because checking the commit merely exists isn’t enough when the orphaned one still exists too. What it cost: on a rebased merge request, a walk of up to a hundred commits where one field read would have done.
Credentials, caller’s choice. The built-in four-step lookup for tokens was deleted outright, and the caller now decides where credentials come from (spec 0005). What it cost: a clean break with no shim, felt downstream, in exchange for a core that no longer depends on a credentials library.
Verify the release commit. GitHub and GitLab both silently ignore the target commit once a tag exists, so the provider verifies it instead of passing it along, and an absent tag is refused rather than created (spec 0013). What it cost: a naming decision reversed after a release had shipped, when short method names collided across all four adapters, and a test now catches that kind of collision.
Merging, but people decide. Merging was first ruled out of the library entirely. That got reversed, on the basis that the house rule is about who merges, not what the code can do (spec 0024). What it cost: “merge when ready” is refused on Bitbucket, where it can’t be checked below a premium workspace.
Linked worktrees just work. Opening a repository inside a linked worktree (the extra checkout git worktree add makes) used to hand back a healthy-looking repository that failed later, with an error about missing objects that never mentioned worktrees. The fix is always on rather than an option, because an option would only exist to let someone ask for the broken behaviour (repo spec 0001). What it cost: a worktree pointing at a repository that’s gone now fails when it’s opened, which is the better place to fail.
Ask for the narrowest part. go/repo is split into ten small roles (reading the tree, committing, branching and so on), so a function that only reads files asks for only that, and a two-method fake can stand in for it in a test. Each role ships a mock, and the build fails if a mock stops matching its role. What it cost: most of go-git isn’t wrapped (fetch, log, merge, status and the rest), so a caller reaches go-git directly through a callback, still under the lock, and gives up the narrow contract while they’re there.

Proof in use

colophon is built on all of it. It uses all four forge adapters, tags its releases from an in-memory clone, and records each release in this blog with a sparse clone of the one directory it writes to, without a disk or a git binary.
phpbotscout uses go/forge and go/repo together. go/forge finds which repositories across the estate have a docs site, by checking each one for its site’s config file, and go/repo shallow-clones each of those into memory so the bot can index the docs.
skillup uses both too. Its GitLab adapter comes through go-tool-base, and every history question it asks and every commit and push it makes goes through go/repo, with nothing shelled out to a git binary. It often runs inside a linked worktree.
go-tool-base uses the forge core, repo and changelog, keryx uses go/forge and go/repo, and krites and sigillum talk to GitLab through go/forge.
183 releases across the forge family since July 2026, 42 of them the core, plus 10 of go/repo, 10 of go/changelog and 6 of aferobilly.
Every forge adapter runs the same conformance suite, and every adapter’s release merge request runs cicd’s go-core-currency check, which flags an adapter about to ship against an older go/forge.

Use it when, and when not to

Use go/forge if a Go tool needs to read or cut releases, or work with merge requests, on more than one forge, or might one day, and you’d rather not carry every vendor’s SDK to do it. Use go/repo if a Go program needs to clone, commit or push without a git binary on the machine, needs to share one repository across goroutines, or wants a real repository in memory for its tests.

Know where the edges are. GitLab is the one proven by daily use; the others are supported by their adapters and tests rather than by me running them in anger. go/forge doesn’t retry, back off or cache, it downloads checksums and signatures without verifying them (that’s go/signing’s job), and some forges can’t answer some questions at all: Bitbucket can’t list releases or find one by tag, and has no issues. go/repo reads files from the current commit only, not another revision, the staging area or uncommitted changes, and named remotes aren’t available on the thread-safe repository. The forge limitations and repo limitations pages have the full lists. Both are pre-1.0, so pin them.

Where it’s going

Most of what’s left is waiting on other people: merge-when-ready on Bitbucket needs a premium workspace to test against, a panic in the Gitea SDK is written up for its maintainers, and a symlink failure in go-git waits on go-git v6, whose pre-releases already fix most of it.

go-tool-base

Mon, 01 Jan 0001 00:00:00 +0000

The problem it exists for

Most of a Go application isn’t the part that’s actually yours. It’s configuration that layers files, environment and flags and can tell you where a value came from, an update you can trust not to install somebody else’s binary, errors a user can act on, a clean way to start, run and stop, and the same handful of commands (init, update, doctor, config, docs) written again for every new thing you build.

go-tool-base writes those once. It’s a batteries-included framework (think Rails or Laravel, for a Go binary), and it started out for command-line tools, because everything starts life as a binary you run. But more than half of what’s built on it today doesn’t exit when a command finishes. keryx and krites each serve a local web studio from the same binary, phpbotscout runs as a long-lived bot, and Scout has a window and a voice.

How it works

The usual commands, already written

A tool built on go-tool-base starts from one shared bundle of the things every command needs (its name and version, a logger, its config, the filesystem, error handling) and adds its own commands, and the framework builds the rest of the tool around them. The built-in commands arrive as features: update, init, MCP, docs, doctor and changelog are on by default, and AI, config, telemetry and man pages are opt-in. A tool switches each one on or off, and the framework behaves accordingly. A tool without an init command, for instance, is allowed to start with no config file at all, because it has no way to create one.

A generator that keeps writing the boring bits

The gtb binary is the generator. gtb generate project scaffolds a new tool, gtb generate command adds a command, and add-flag, regenerate, remove, enable and disable keep working on the tool after day one. gtb enable signing, for example, wires in signed self-update with the verification turned on.

Long-running, too

A tool that needs to stay up gets the same supervisor the framework’s service commands use, controls: concurrent startup, health and readiness probes, shutdown in reverse order of registration, and restarts when something falls over. That’s what sits under a studio served from the CLI, or a bot that’s meant to run for weeks, and the config, update and signing story is the same one the short-lived commands get.

Made of modules, not a monolith

Most of what go-tool-base does lives in separate toolkit modules it requires: config, errors, transport, observability, forge for releases and self-update, signing, MCP and chat. go-tool-base is assembled from 45 of them, and the toolkit they come from now holds 93 public modules, nearly all of them split out since the extraction playbook was written in mid-July 2026. You can use any of them without go-tool-base, and go-tool-base is what makes them agree with each other.

Decisions and what they cost

Split into modules. The framework was broken out into modules one package at a time, following an extraction playbook (spec 0119), and the generator itself became a nested module of its own (spec 0194). The payoff was measurable: a minimal binary went from 81.3 MB to 34.3 MB, 58% smaller, once the adapters stopped coming along for the ride. What it cost: shared test harnesses get copied into each module, and the generator’s own packages are importable but not supported.
Off Viper. Config moved from a Viper wrapper to go/config’s store (spec 0136). What decided it was coherence: reading from a frozen snapshot means a set of related reads never mixes values from before and after a reload, and config’s own test holds that at zero mismatches. A forwarding adapter that would have kept the old API was turned down because it brought the mismatch straight back. What it cost: roughly 350 call sites and 595 lines of documentation, rewritten.
An errors package of my own. I’d originally chosen cockroachdb/errors (spec 0001), and later reversed my own decision: the estate used ten of its functions, and paid 46 of go/forge’s 376 transitive packages for them. go/errors replaced it, with hints for the user built in (the reason an alternative without them was passed over). What it cost: one change touching 116 files and 50 sentinel errors.
Signed self-update. An update is verified against a signature before it’s installed (spec 0056), using GPG with keys published over WKD, chosen over cosign so verification still works offline and behind a firewall. What it cost: a compromise of both the forge and the key directory at once wouldn’t be detected until a transparency log lands, and that’s still a draft.

Proof in use

Seven tools are built on it today, all on the same release: keryx, krites, sigillum, skillup, phpbotscout, colophon and Scout.
66 releases since v0.1.0 in May 2026, and the version in the header above is the current one. A tool’s release merge request runs cicd’s go-core-currency check, which flags a release still built on an older go-tool-base. All seven, and go-tool-base itself, build and publish their binaries through cicd’s goreleaser component.
290 behaviour scenarios across 52 feature files, and 2,954 test and fuzz functions, with a 90% coverage threshold on the pipeline.

Use it when, and when not to

Use it if you’re building a Go application that real people will install and keep updated, whether that’s a command-line tool, something that runs for weeks, or a local web UI served from the same binary, and you’d rather not write config layering, signed self-update, graceful shutdown, a doctor command and an MCP server for the fourth time. Release builds cover Linux, macOS and Windows on amd64 and arm64.

It isn’t a web framework, though. It won’t route your requests or render your pages, and it won’t scaffold a microservice for you: what a long-running process gets from it is the lifecycle around the work, not the work. And it’s overkill if all you need is a command router. It’s also pre-1.0: breaking changes still ship as minor releases and the stability tiers aren’t in force yet, so pin the version.

Where it’s going

The static release channel (spec 0203) has shipped. A tool can find and fetch its own releases from a plain HTTPS location, with no forge in either step, and an updater that needs nothing but HTTPS and its embedded keys is one that can repair a broken install. keryx is the next tool to move onto it. Behind that, update channels and rollback (spec 0104) are drafted, and need another look now that the channel lists every version (go-tool-base#117).

keryx

Mon, 01 Jan 0001 00:00:00 +0000

The problem it exists for

One person writes a blog post. Then it wants a cover, perhaps a narrated vertical reel with music of its own, and a differently shaped post for each social network, each one sent on the right day. Doing that by hand for every article is a second job.

keryx turns it into commands, and the blog’s pipeline runs them unattended. The name is ancient Greek for a herald, which is precisely the job. It has made this blog’s reels since June 2026, and has sent its social posts on schedule, across six platforms, since 20 September 2026.

How it works

Everything is a file in your repository

It’s one Go binary that keeps no state of its own. Reels, themes, schedules and the record of what went where all live in the project’s own git repository, so a cover, a storyboard or an approval is something you can review in a diff.

Generate several, pick one

Every step that costs money generates a few candidates and lets you choose: covers in 16:9, and reels as 9:16 text cards over generated images, with narration in a cloned voice and a music bed to match. A local web studio covers writing, previewing, approving and publishing, and making a whole reel in one pass asks before it spends anything.

Approved, then sent

Social copy is written per platform, because a Bluesky post isn’t a LinkedIn post. It can send to nine networks, and the blog uses six: Bluesky, Instagram, Facebook, LinkedIn, Threads and Mastodon. Nothing goes out without a human approval and a due hour, and an hourly job sends whatever has come due.

Decisions and what they cost

Render in memory. Reels render through afmpeg, so any project can render without FFmpeg installed or a local checkout (spec 0021). What it cost: the portable engine took about eight and a half minutes on a short three-card reel where native FFmpeg takes seconds.
No themes in the binary. keryx ships with no themes at all (spec 0046), on the principle that the seed is code and themes are content, so a change of taste never needs a release. What it cost: a fresh install can’t generate anything until you bring a theme of your own.
A studio that locks itself. On localhost the studio is open; bound to any other address it mints a one-run token as a strict cookie, and refuses to start if it can’t build that gate (spec 0018). What it cost: a cookie verifier had to go into go-tool-base first.
Mastodon instead of X. Bluesky got a small hand-written client, and Mastodon took X’s place in the default set because X’s API now charges per use (spec 0060). What it cost: an X adapter that’s built and tested and sits unused.

Proof in use

This blog’s social posts have gone out through it since 20 September 2026, and its reels since June 2026.
31 releases since June 2026, built on go-tool-base, writing copy through chat, and rendering with afmpeg.

Use it when, and when not to

Use it if you publish regularly and want covers, reels and social posts made and scheduled from the same repository as the writing, with you approving every one.

It’s built to swap providers, but for now each capability ships with one: Gemini for images, ElevenLabs for voice and music, five options for the chat that writes the copy. It isn’t a hosted service, there’s no video generation, reels have a fixed shape (no editing timeline, nothing long-form), card text is Latin script only, and a LinkedIn sign-in lasts about sixty days before you renew it by hand. The limitations page has the rest.

Where it’s going

Grounding the social copy more firmly in the article it’s about, and a static release channel for its own updates, are next.

krites

Mon, 01 Jan 0001 00:00:00 +0000

The problem it exists for

A full wedding day comes back as three or four thousand frames, and every one of them has to be judged before any editing can start. By hand, that can take a photographer weeks. krites was built for one photographer in particular, my wife, who had four thousand frames and a weekend to get through them (I built my wife a judge).

And a wedding’s photos are sensitive, so the whole thing runs on her own laptop: no subscription, it works with no signal, and nothing leaves the machine unless she switches a cloud option on for herself.

How it works

A first pass, with reasons

Point it at a folder and it marks every frame keep, maybe or reject, each with a reason in plain words. Blur, exposure and near-duplicate bursts are judged with plain arithmetic (There’s no AI in my photo culler), and eyes and expressions with small models that run locally.

Review in a studio on your own machine

The review happens in a browser studio that only ever runs on your laptop. The originals are never written to: every decision is kept as a record beside the shoot, only an export writes new images, and sidecar files carry the cull straight into Lightroom (ten lines of XML).

Decisions and what they cost

Licence over benchmark. The most accurate expression models were trained on a dataset whose terms forbid commercial use of anything derived from it, so krites uses MediaPipe’s face models instead, converted to run locally (spec 0029). What it cost: about a fifth of faces got no expression reading at first, tuned down to 15%, which is acceptable because a face with no reading never causes a wrong reject.
A better detector. YuNet replaced the old face detector as the default rather than outright (spec 0033). On a real wedding it found 718 faces against 636, 13% more, with no change in rejects. What it cost: switching detector means re-culling the shoot.
Opt-in removal. Removal runs locally with LaMa, or MI-GAN as an alternative, turning down diffusion models and anything whose own licence forbids commercial use (spec 0008). What it cost: both models were trained on a dataset with non-commercial terms, so removal stays switched off until the user explicitly accepts that.
No learning yet. Learning the photographer’s own taste was parked until there’s real use to learn from (spec 0036), because with no human corrections yet, picking a learning goal would be a guess dressed as a design. What it cost: it doesn’t learn yet, and most frames still land on “maybe”.

Proof in use

It was built after a real fourteen-hour wedding of more than four thousand frames, and its detector change was proved by re-culling a real shoot.
Blink detection from the shape of the eye catches 12 of 14 real blinks, against 4 of 14 for the trained model it was compared with.
19 releases since June 2026, and its models and runtime come through Secure artefact delivery, signed and checked like everything else.

Use it when, and when not to

Use it if you shoot events, come home with thousands of frames, and want a first pass you can understand and overrule, without your clients’ photos going anywhere.

It isn’t a catalogue or a RAW developer (a RAW frame exports as a JPEG), it works on one shoot at a time, there’s no Windows build, and the Mac download is Apple Silicon only. It doesn’t score aesthetics or learn from you yet, ingest skips subfolders and some newer RAW formats, and the optional AI review sends the full image to a third party. The limitations page has the rest.

Where it’s going

Learning from real corrections once there are enough of them, finishing the expression pipeline’s tuning, and replacing the removal models with ones whose training data is clean, which is still an open research question.

mcp

Mon, 01 Jan 0001 00:00:00 +0000

The problem it exists for

The obvious way to put a command-line tool behind MCP is one tool per command, and every one of those tools brings its schema into the agent’s context before the conversation even starts. The feasibility spike measured how that grows: about 16.7 KB for ten commands, 167 KB for a hundred and 1.67 MB for a thousand, against a fixed few hundred bytes for a small three-tool front door (the spike). On a real build, go-tool-base’s whole listing through that front door is about 1.5 KB.

How it works

Three tools in front of everything

go/mcp puts one registry behind three discovery tools: search, details and call. An operation’s schema only reaches the conversation when an agent actually asks for it. A direct mode, publishing one tool per operation, is there when a client really wants it.

Commands, HTTP and gRPC behind one registry

Command-line commands run as supervised subprocesses of the same binary, an HTTP service mounts a single handler, and gRPC methods bind in-process through the service’s own interceptors. Being listed in the catalogue isn’t permission: the policy is asked again at every step, and a denied operation answers exactly like an unknown one.

Decisions and what they cost

Compact by default. The three-tool front door is the default and direct publishing has to be chosen (spec 0001), rather than guessing the mode from how a client behaves. What it cost: a client’s approval prompt only ever sees the call tool, which has to carry the most cautious labels.
Commands run out of process. A command is run as a fresh subprocess of the tool, never in-process, because a command tree is shared mutable state. What it cost: one call at a time (a second gets an immediate, retryable busy), a five-minute limit, a megabyte of output, and stricter checking of custom flags.
Permission at every step. The catalogue and execution are shared by every publishing mode, and the policy is checked on each request, not just once at listing.
gRPC in-process. gRPC calls go through the service’s own interceptors in-process, rather than dialling the service’s own listener, which would need a service-to-itself token (spec 0002). What it cost: single request and response only; streaming methods are refused when they’re registered.

Proof in use

go-tool-base serves its MCP endpoint through it, so every tool generated on go-tool-base with MCP switched on gets the front door for free.
It’s tested against three versions of the official Go MCP SDK.

Use it when, and when not to

Use it if you want AI agents to be able to drive your Go tool’s commands or services, and your tool has enough of them that publishing every schema up front would crowd the conversation.

It doesn’t authenticate callers, and it doesn’t sandbox commands beyond a process group, so one mode of it isn’t suitable over HTTP at all. The publishing mode is fixed when the server is built, and cleanup on macOS and Windows is designed and compiled but not yet proven in CI.

Where it’s going

keryx’s studio as a second user, native macOS and Windows acceptance, gRPC streaming, and a bridge into chat.

Media in pure Go

Mon, 01 Jan 0001 00:00:00 +0000

The problem it exists for

keryx makes its promo reels with FFmpeg, and it used to do that by running the ffmpeg program, which needs real files on a real disk. So a project held only in memory couldn’t be rendered at all. The usual ways round that each had a catch: pure-Go bindings weren’t mature, CGO bindings break static cross-compiles, and the existing WebAssembly builds of FFmpeg were missing the crossfade filter and AAC audio, had nowhere to put files, and were pinned to FFmpeg 5.1, which no longer gets security fixes.

So the goal was FFmpeg in a Go program with no CGO, nothing installed on the host and no temp files. It’s two projects. ffmpeg-wasi builds the engine, and afmpeg runs it from Go.

How it works

ffmpeg-wasi: current FFmpeg in a sandbox

ffmpeg-wasi compiles FFmpeg’s libraries to WebAssembly and drives them with an engine of its own, which takes a structured job (inputs, filters, outputs) instead of an ffmpeg command line. What comes out is a single .wasm file that runs on any machine and any processor architecture with a WebAssembly runtime, and it tracks current FFmpeg (n9.0.1 today). That matters for a library whose whole job is reading media from strangers.

The sandbox is the point. The engine sees only the files it’s handed, and nothing else on the machine. Its memory is capped, so a crafted file that claims enormous frame sizes and asks for gigabytes gets an allocation failure inside the sandbox, and the job fails. Without the cap that allocation lands in the host process, the kernel kills it, and every other request in flight goes with it.

afmpeg: running it from Go

afmpeg runs that engine inside the Go program under wazero, a WebAssembly runtime written in Go. Every file the engine opens is answered from a filesystem the caller hands it, an in-memory one or anything else, and each job gets fresh in-memory scratch space, so FFmpeg believes it has a disk and never touches one (FFmpeg thinks it has a disk). The limits are on before you ask: 512 MB of memory, an hour per job, and one job at a time per runtime.

The native driver: same engine, native speed

The sandbox has a price. WebAssembly here means one thread and no SIMD (the processor instructions that do many sums at once), and video encoders lean on both. So ffmpeg-wasi also compiles the same engine as a native Linux program with real threads and SIMD, and afmpeg runs it as a separate process, still without CGO. The caller’s filesystem is served to it over a local socket, with reads, writes and seeks each sent as a message, so even the way an MP4 writer jumps back to patch its header goes through the caller’s filesystem. Nothing touches the host disk on this path either.

The custom driver is what makes this work. Against the sandboxed engine it encodes about 50 times faster with openh264 and about 170 times faster with libx264. Against the host’s own ffmpeg program, on the same jobs, it took between a fifth of the time and slightly under the same time (the speed report). That comparison isn’t exact: the host program was an older FFmpeg, and it used temp files where the driver stayed in memory. The fair reading is that the socket bridge and the no-disk guarantee cost nothing measurable, and the driver runs at native speed. It also carries the heavy HEVC and AV1 encoders that are impractical in WebAssembly.

The driver isn’t a sandbox, so it’s fenced in instead. On Linux the kernel’s Landlock file restrictions limit what it can open, and any access it can’t account for is fatal rather than silent.

Signed from source to runtime

Every input to the build is pinned: the toolchain image, the FFmpeg tag, and each codec library by tag, commit or checksum. Downloaded source has to match a hard-coded checksum, and a mirror that disagrees is rejected. Each release publishes a checksum file covering every .wasm module, every native driver and the build record, and one OpenPGP signature over that file certifies the lot.

The signing key lives in AWS KMS and never leaves it, and only this project’s release-tag pipeline can use it, by proving its identity to AWS rather than holding a credential (signing and trust covers how). afmpeg carries the public key inside its own code and cross-checks it against a copy published on phpboyscout.uk, which GitLab doesn’t control. It checks the signature and the checksum before an engine or a driver is written to disk or run, so a program using afmpeg never runs an engine it hasn’t verified.

Decisions and what they cost

Libraries, not the program. Linking FFmpeg’s libraries into an engine of its own beat compiling the ffmpeg command-line program, which would have meant either an end-of-life FFmpeg or a multithreaded version the WebAssembly runtime can’t run (spec 0007). What it cost: a C engine and a versioned job vocabulary to own, and ffmpeg command lines aren’t accepted.
Licences kept apart. The Go package stays MIT, and the engine is a separate, signature-checked download that’s never embedded, so FFmpeg’s GPL or LGPL terms land only on whoever chooses and bundles an engine (spec 0001). What it cost: you have to pick and fetch an engine, and without one the library won’t start.
Native, but no CGO. The fast path is the same engine in a separate process rather than FFmpeg linked in through CGO, which would taint the MIT licence and lose the static cross-compile (spec 0028). What it cost: a socket protocol to own, the memory cap and engine progress don’t apply to the driver, and it’s published for Linux on amd64 only.
Fence the native driver. After two independent architecture reviews threw out two earlier designs, the native driver’s file access is restricted by default and any access it can’t account for is fatal (spec 0043). The escape hatch that turned out to be french doors is how that came about.

Proof in use

keryx renders its reels with it, fetching the engine as a signed download.
22 releases of afmpeg and 20 of the engine, which is on FFmpeg n9.0.1. Every engine release, native drivers included, is covered by one signature.
The speed gap was re-measured across a major FFmpeg version and didn’t move, because it comes from WebAssembly’s single thread and missing SIMD rather than from FFmpeg.
The lean engine is about 5 to 6 MB to download.

Use it when, and when not to

Use it if a Go program needs to transcode, mix or cut media without asking anyone to install FFmpeg, and without CGO or temp files. Reach for the sandboxed engine when the media comes from people you don’t trust or the program has to run anywhere, and for the native driver when speed matters and you’re on Linux.

The sandboxed engine is slow on encodes, roughly 50 to 170 times slower than the native driver, which is fine for short clips and a reason to switch for anything heavy. Decoding and filtering cost far less. There’s no network streaming, GPU encoding or capture devices, HEVC and AV1 encoding exist only in the native driver, jobs run one at a time per runtime, and the native driver is Linux amd64 only. The limitations page has the rest.

Where it’s going

Better audio and video sync is in review. Multithreading inside the WebAssembly engine, progress reporting from the native driver and a command-line tool for people who’d rather not write Go are all drafted and parked, each until something needs it.

messaging

Mon, 01 Jan 0001 00:00:00 +0000

The problem it exists for

A service that talks to another over a Go channel has to be rewritten the day the other end moves out of the process. An earlier attempt at a message bus here was designed directly against NATS, and it showed: a subscriber that panicked took the whole process down with it.

The deeper lesson was about abstraction. One with a single implementation is shaped entirely by that implementation, and an in-memory backend that copies NATS proves nothing about anything that isn’t NATS (spec 0001). So the contract was treated as provisional until something genuinely different arrived, and when SQS, AMQP 1.0 and RabbitMQ were specified, they forced nine changes to the core contract before any of them could be built.

How it works

Services talk to the bus, never the transport

A service imports the bus and CloudEvents, and nothing else, so moving from in-memory to NATS or a cloud queue is a choice of backend, not a rewrite. Under the five modules sit seven backends: in-memory, core NATS, JetStream, SQS FIFO, SQS Standard, RabbitMQ and AMQP 1.0. As well as publish and subscribe, a service can ask a question and wait for one answer, and every request is counted by how it ended. go/nats embeds a NATS server in the process, for tests and single-binary deployments.

Bounded, counted, and declared

Every subscription chooses how hard delivery tries. The default is at most once: one attempt, and a failure is counted rather than retried. At least once retries until the handler acknowledges, up to a limit, under a name the broker recognises after a restart. Asking for it on a backend that can’t survive a restart is refused unless the subscription says that’s fine. Every subscription has a size limit, and every message dropped at that limit is counted against the subscription it was meant for. Anything a backend can do beyond the basics it has to declare, and one shared conformance suite holds every backend to exactly what it declares.

Decisions and what they cost

The unit is a CloudEvent. Messages are CloudEvents, not raw bytes or any Go value. What it cost: anything that can’t be serialised can’t travel. Audio and video can, as bytes in an event’s data, but not efficiently. Each frame carries its own envelope and is copied once per subscriber, frames dropped at a full queue can’t be asked for again, and order holds only when one handler takes one message at a time (the limitations). A direct stream does that job better.
Bounded queues. Every queue is bounded and every drop counted, but what happens at a full queue is each backend’s to offer, rather than a queue bolted in front of every backend to make the options mean the same thing everywhere. What it cost: swapping backend can change what happens when a queue is full; core NATS can only refuse the newest message.
Generality has to be proven. Three backends unlike NATS were specified before NATS itself was built, with a shared conformance suite to keep them honest. What it cost: the nine contract changes, an SQS race suite that takes about eleven minutes, and a test image pinned to the last version that runs without an auth token.
Declare the edge case. JetStream declares that a server restart can use up a delivery attempt nobody saw (spec 0012), instead of hiding it or trying to rebuild the count from server advisories. What it cost: the count of exhausted messages can come up one short, measured at 11 runs in 250.

Proof in use

phpbotscout and comms-discord run on it, and so does Scout, its first user of request and reply.
The conformance suite runs five deliberately lying backends and requires every one to fail, after an audit found a lying backend could pass ten green cases. SQS Standard passes all 31 of its rows.
22 releases across the bus and its four backends since September 2026, under controls for its lifecycle. A backend’s release merge request runs cicd’s go-core-currency check against the bus’s latest release.

Use it when, and when not to

Use it if your Go services need to talk asynchronously and you’d like to start in memory and move to NATS or a cloud queue later without rewriting them.

Nothing promises exactly once, so a handler subscribed at least once has to tolerate duplicates. A subscriber sees its messages in order only while it reads one at a time. Give it more readers and order goes, and nothing orders messages across subscribers. A message a handler turns down is redelivered straight away, with no backoff yet, so a handler waiting for a dependency to recover has to do its own waiting before it says no. Nothing routes traffic around a struggling consumer, and it isn’t a universal broker abstraction: anything past the two portable delivery shapes is a capability a backend declares. go/nats runs an embedded server in-process only, because it has no TLS or authentication for a listener yet, though a client connects to a cluster someone else runs just fine. go/schema, the schema registry, is at its first release and says itself it isn’t obviously needed yet.

Where it’s going

Backoff on redelivery, capabilities that depend on the connection, a deduplication window for embedded NATS, and keeping the message envelope swappable.

phpbotscout

Mon, 01 Jan 0001 00:00:00 +0000

The problem it exists for

Support questions arrive on Discord, get answered from memory, scroll away, and get asked again a week later. The problems worth tracking rarely become GitLab issues, and quite often the answer was already sitting in the docs.

phpbotscout answers what the docs already cover, with citations, and treats every question it can’t answer as “a documentation gap with a timestamp on it”. Picking its model was a costed decision: on 36 calibration questions run through the real answer path, the small model matched the big one closely enough that the difference came down to about £5 a month at fifty questions a day, against £38 (spec 0009).

How it works

Answer only from what it was shown

It reads the public channels, searches an index built from the estate’s public docs and code, and hands the model eight passages per question. The model answers only from those passages, and an answer that cites nothing, or cites something it was never given, is turned into a decline instead (how answering works). The most tempting corpus in any repository is about what went into that index and what didn’t.

Discord behind a contract

Discord sits behind go/comms, a contract for chat platforms with no vendor SDK in its core, and go/comms-discord is the provider that speaks Discord.

Decisions and what they cost

The small model, for now. Haiku 4.5 scored nearly as well as Opus and was the fastest of the three. Sonnet was turned down at four times the cost for identical scores and the worst slow-case latency, and Opus at 7.6 times the cost, partly because it writes so much more (spec 0009). What it cost: the choice is provisional, because citation accuracy, which the spec calls the worst failure there is, wasn’t measured.
A library that resumes. disgo was picked because it was the only one that resumes on the reconnect address Discord gives it, proved live by replaying messages posted during a twenty-second outage (spec 0004). What it cost: the heaviest dependency footprint of the three, a slower connect, and a library that’s still below 1.0.
Quiet until a server opts in. By default it composes answers without posting them, and a server only gets posts if its own setting says so. A server that hasn’t opted in gets a bot built read-only, with no way to post at all (spec 0025). What it cost: that setting had to be restructured per server.
Logs carry no message text. No message text or author ends up in a log line, so retention stays where the policy enforces it (spec 0028). What it cost: the per-message trace is gone.

Proof in use

It’s been running in shadow mode since September 2026, reading every public channel and composing answers, and posting only where a server has opted in.
Its index covers 2,025 documents from the estate’s public projects.
A round of tuning across 142 real questions cut the ones with no answering passage from 22 to 12, and lifted the questions where the top passage answered it from 61 to 76.
19 releases of the bot since August 2026, built on go-tool-base and answering through chat.

Use it when, and when not to

Use it as a pattern if you run a community around your own docs and want questions answered from them, with sources, without letting a model make things up.

It doesn’t moderate or raise GitLab issues yet, it works for one project’s servers, only in public channels, with keyword retrieval rather than semantic search, and only with Anthropic models. Answers are only as current as its last refresh, and releases are checksummed but not yet signed.

Where it’s going

Following up inside a thread and a way for a member to ask to be forgotten are both drafted. Local inference has landed but isn’t used yet, and the hybrid retrieval that would use it, finding a page that answers a question even when they share no words, is in review. Posting everywhere waits on thirty human-reviewed answers to real questions (The bot couldn’t answer it. Neither could I.).

rust-tool-base

Mon, 01 Jan 0001 00:00:00 +0000

The problem it exists for

A confession first. rust-tool-base started as a vanity project and a way of learning Rust properly, begun not long after go-tool-base went open source, back when go-tool-base was still a single monolithic project. Since then the Go side of the estate has grown enormously and kept accelerating, and the Rust side couldn’t keep pace, so it’s lagging behind. It isn’t forgotten, though. It’s still very much on the radar, and it’ll get more love as soon as there’s more time (and more runner capacity and AI quota) to give it.

With that said, here’s what it’s for. Rust has superb single-purpose crates and no ready-made way of putting them together into a tool people install and keep updated: configuration, self-update, docs, credentials, AI, MCP and telemetry, all agreeing with each other. That’s the gap go-tool-base fills for Go, and rust-tool-base fills it for Rust.

It isn’t a port, though. “Same idea, different idioms” is the whole point: the same outcomes for a tool’s users, reached the way Rust prefers, which turns out to be quite a different shape. Its spec carries a ten-row table of the places the two part company.

How it works

What’s shared, and what isn’t

The two frameworks share outcomes, names (the forge and chat crates were renamed to match their Go modules), the minisign signature format and the shape of a repository. They don’t share code, a runtime, or even a config format: Go’s tools read YAML and Rust’s read TOML.

Where they differ is where Rust earns its keep. Configuration is typed instead of looked up by string, builders refuse to compile when a required field is missing, commands register themselves at link time, and errors are values you can match on. A tool is its commands, a registration and one call to build the application.

One crate per idea

Each library is its own crate in its own repository under rust, all published to crates.io with the rtb- prefix, and an umbrella crate switches them on with feature flags and pins a set of versions known to work together. It releases with release-plz rather than colophon, for now: colophon#37 is working out what it would take to change that.

Decisions and what they cost

Twelve small repositories. The framework was split into roughly a dozen repos, one concept each, over about six larger ones or Go’s one-repo-per-plugin layout (spec 0043). Before the split, one merge request’s pipeline ran about 51 minutes of compute. What it cost: a dozen docs sites, more load on the runners, and version skew that shows up as two copies of one crate in a build, which makes automated updates and a duplicate-crate ban part of correctness rather than tidiness.
Its own config store. Config is moving to a native store of its own, replacing figment, with string getters refused for good (spec 0044). What it cost: a hard break, taken now because there’s nobody downstream to break yet.
Go’s design, Rust’s wiring. Forge access follows go/forge’s vendor-free design, but providers are passed in explicitly instead of through a global registry (spec 0042). What it cost: splitting the forge and chat crates per provider was deferred, so the forge crate still carries all six backends.

Proof in use

Every crate is on crates.io, none yanked, with the umbrella at 0.9 and the libraries between 0.6 and 0.9. Its release binaries are signed with minisign through KMS, the same way as everything in Signing and trust.
31 posts so far, starting with the same idea, in a language that argues back.
Nothing outside the family depends on it yet, which is what you’d expect of the part of the estate that’s been waiting its turn.

Use it when, and when not to

Use it if you’re building a command-line tool in Rust and want config, self-update, docs, credentials and AI wired together the way go-tool-base does it for Go, and you’re comfortable being an early adopter.

Don’t use it yet if you need stability: it’s pre-1.0 and the API isn’t frozen, release binaries are Linux x86_64 only, and the limitations page lists seventeen gaps, nested subcommands among them. Like its Go sibling, it’s not a web framework.

Where it’s going

Catching up, as time, runner capacity and AI quota allow. The new config store is approved and still to be built (rust/config#10), and splitting the forge crate per provider comes after that (rust/forge#9).

Secrets in use

Mon, 01 Jan 0001 00:00:00 +0000

The problem it exists for

Every tool that talks to a service needs a token, and the easy place to put one is the config file. That’s the threat go/credentials is written against: “a plaintext secret committed to, or left in, a config file. Config files get checked into git, copied between machines, attached to bug reports, and synced to backups.”

The second problem is quieter. Once a token can come from several places (an environment variable, the OS keychain, a value in the file), something has to decide which one wins. When every library decides for itself, a program ends up with several precedence orders stacked inside each other, and nobody can say which token it’s actually using. The forge library had exactly that, and its spec put it plainly: “Two precedence systems in one path is how configuration becomes unpredictable” (forge spec 0005).

How it works

A credential is a source, not a string

A library that needs a secret takes a function it can call when it needs one, never a string handed over at start-up. The secret is looked up at the moment of use rather than carried around, and the library doesn’t care where it came from. Returning nothing means “not configured”, and returning an error means “configured but broken”, so a keychain that won’t unlock never reads as a missing token. Git and forges and go/nats take credentials this way, and the reason is in nats’ own words: “precedence is the consumer’s decision, not a library’s.”

The order is decided once, by the application

go-tool-base states its order in one place: an environment variable it’s been told the name of, then the keychain, then a value in the file, then the variable a vendor’s own tools would use. Both the code that supplies a token and the doctor command that reports on it build from that one statement, so the two can’t disagree about which source wins (spec 0189). doctor shows what’s stored, where each credential resolves from and what’s being shadowed, and none of that report is a credential value. Each source is only consulted when it’s needed, so a repository reached over SSH never triggers a keychain prompt for a token it doesn’t need.

When a keychain entry is configured but can’t be read, it refuses rather than falling back to a plaintext value in the file. The default storage choice reads the environment too. It uses the keychain only outside CI, at a real terminal, and after a test write succeeds. Everywhere else it records the name of an environment variable.

Where to keep it

go/credentials is the storage half. A config file records one of three things: the name of an environment variable, a keychain reference, or, as a last resort, the secret itself. Storing the secret itself isn’t offered in CI, and is refused there if it’s chosen anyway. The OS keychain is opt-in: leave it out and the keychain libraries never enter the binary, which you can prove from the build’s software bill of materials. The module deliberately has no resolver and no precedence order, because “the order in which your tool consults those sources at runtime is your tool’s policy, and it is written in your tool.”

config treats secrets stores as layers marked sensitive. The secrets managers (Vault, AWS, Google Cloud and Azure) are read-only, and a write that would copy one of their values into a plain file is refused. The keychain layer is the one sensitive layer you can write to: a token goes into the keychain, and one already there can never be written to the file beneath it. A shared test suite holds every sensitive backend to that rule.

go/vaultclient builds a Vault client four ways, from your own client to the ambient environment, and never logs the token at any level. A test sets a token and fails if it turns up in the output.

Scrubbed on the way out

go/redact masks secrets in text before it leaves the process: passwords in URLs, name=value pairs and JSON fields with credential names, authorisation headers, JWTs, known token prefixes from GitHub, GitLab, AWS, Google, OpenAI and Slack, and long opaque tokens. Its rule is “Sanitise where data leaves the host, not everywhere.” A second pass changes nothing, so an outer boundary can scrub again “without coordination”, even when an inner one already has (threat model). transit runs every outbound URL it logs through it, and always masks sensitive header values, even when a caller asked for them to be logged. The rules can’t be switched off, because “a redactor whose rules can be weakened by the environment is a redactor that can be switched off by accident.”

And in the pipeline

gitleaks runs in every language track’s security component, scanning only the commits a merge request adds, with its own output redacted: go-security on 99 projects, rust-security on 12, tofu-security on 7, and the svelte and skills components. 119 of the 139 projects with a pipeline run at least one.

Decisions and what they cost

No ladders in libraries. forge’s built-in precedence was removed and the order moved into the application (forge spec 0005). What it cost: config reads every backend when it starts, so a program built on config alone unlocks every credential at start-up whether it uses them or not. go-tool-base avoids that by making each source lazy.
A function, not an interface. Every real credential source is a one-liner, and an interface would make each one declare a type. What it cost: go/nats and go/forge each define the same function shape rather than sharing one.
The caller’s deadline. A slow keychain is abandoned when the caller gives up, and there’s no default timeout because “no single value is defensible” (spec 0001). What it cost: an abandoned save can still finish after you’ve moved on.
Fixed redaction rules. What it cost: false positives such as sort-key=***, and short or bespoke secrets get through.

Proof in use

go-tool-base, keryx, krites and phpbotscout store credentials through go/credentials. go/redact is used directly by go-tool-base, phpbotscout, forge, transport and transit, and by 16 more modules through them.
7 releases of go/credentials since July 2026, 6 of go/redact, and 14 of the keychain config layer.
The guarantees are tests: the keychain stays out of the core’s dependency graph, Vault’s token never reaches a log, and redaction is fuzz-tested.

Use it when, and when not to

Use it if you’re building a Go tool other people install, and you want their tokens out of config files and out of your logs, without picking a credential order on their behalf.

It isn’t a secrets manager. go/credentials doesn’t rotate, cache, retry or audit, and holds one keychain backend per process. go/redact recognises shapes, not randomness, so a short secret with no telling name gets through, and it doesn’t parse YAML or JSON. A Vault token doesn’t renew itself, so a long-running process fails once it expires. go/errors doesn’t redact on purpose: keeping values out of errors is a convention, and go/redact is applied at the boundary. One place doesn’t follow the rule yet: go/chat still looks a provider’s API key up through its own fixed order.

Where it’s going

Holding a credential that expires and has to be renewed in a long-running process (go/credentials#5), OAuth and SSH credentials in go-tool-base (go-tool-base#121), keryx moving its secrets into the keychain layer (keryx#7), go/chat handing its key order to the application (go/chat#35), and transit’s docs catching up with the masking it already does (transit#7).

Secure artefact delivery

Mon, 01 Jan 0001 00:00:00 +0000

The problem it exists for

A tool that needs a machine-learning model or a native runtime usually downloads it from the vendor’s URL and checks it against a SHA-256 that somebody pasted into the source. That digest can’t be rotated or revoked, it says nothing about who published the file, and it doesn’t scale. As go/onnxruntime’s docs put it, a table of four platforms across three versions is twelve constants, and “in practice one person verifies the first and copies the pattern”.

The channel carries twelve files today, ten runtime archives and two models. Each of those would otherwise be a hand-typed constant in every tool that uses it.

How it works

Approve it once, in a merge request

Adding an artefact to the channel is a merge request against one list. Once it merges, the pipeline fetches the file from upstream, checks it, records its digest and size, signs a manifest for it with a KMS key it reaches through OIDC (so there’s no stored credential anywhere), and publishes it.

Check the signature before downloading

go/artifacts is the client a tool uses, and it checks three things in order, the first two before it downloads the file at all:

a signed index, which has to list that exact version as approved
the manifest’s signature, against both the key built into the tool and the key published over WKD, which have to agree
the file’s digest and size, once it arrives

go/onnxruntime sits on top for the ONNX Runtime specifically, resolving the right archive for the platform. There isn’t a pasted digest anywhere in it.

Decisions and what they cost

A manifest per version. Every artefact-version gets a manifest of its own, signed and published when its merge request lands, instead of one manifest over everything cut from repository tags (spec 0014). What it cost: the signing key now trusts the main branch, not just tag pipelines, so an approved merge can mint a signature.
Exact versions, no “stable”. Tools pin exact versions. A floating “stable” pointer was turned down, because a moved pointer still verifies, so nobody would notice it had moved. What it cost: every upgrade is a deliberate, approved bump.
The index is required. There’s no fallback mode that skips the signed index when it can’t be reached, and an index expires after 24 hours (spec 0016). What it cost: the host is a hard dependency. When the job that keeps the index fresh wasn’t running, the channel failed closed for about a day and a half, which is the design doing its job, with every pipeline still green.
One bucket, on a neutral name. The channel moved to object storage on a hostname that names no provider (spec 0018). What it cost: the location is part of what’s signed, so the move meant re-signing everything and waiting a day for the old indexes to expire.

Proof in use

krites loads its runtime and its face models through it: ONNX Runtime 1.23.0 through go/onnxruntime, and the MediaPipe face models through go/artifacts.
The signed index is reissued continually, because a tool refuses one older than a day. It was at generation 155 on 10 October 2026.
The signatures use the same keys and the same WKD check as everything else in Signing and trust.

Use it when, and when not to

Use it as the pattern if a tool of yours downloads something large and native at runtime, and you’d rather it proved where the file came from than trusted a constant.

It’s deliberately narrow. It never picks a version for you (no “latest”, no listing), it doesn’t unpack or load what it fetches, and it can’t protect a file once it’s handed you the path. An outage of the host stops a fresh resolve, withdrawing an artefact can take up to a day to reach a tool, and it vouches for what was published, not that upstream wasn’t compromised before that. go/onnxruntime has no Windows build yet.

Where it’s going

Finishing the move to object storage, which means retiring the old package registry copies and moving the last image that still fetches from there. The specs that define the channel are also due to move from phpbotscout’s wiki, where it started, to the channel’s own.

Signing and trust

Mon, 01 Jan 0001 00:00:00 +0000

The problem it exists for

Most projects publish a checksum file next to their binaries and call it verification. It catches a download that got corrupted on the way, and that’s about all it catches, because whoever can swap the binary can swap the checksum file sitting beside it. The spec that started all of this puts it in one line: “If an attacker can replace the binary, they can also replace checksums.txt.”

So the question isn’t how to publish a checksum. It’s how to prove a release came from you, when the place you publish it is exactly the place an attacker would go.

How it works

Sign with a key nobody can copy

sigillum signs a release in one of two formats: an OpenPGP signature over the checksum file, which gpg checks, or a minisign signature beside each file, which minisign checks. The key lives in a key service and never leaves it: sigillum sends a digest of what it’s signing and gets a signature back, so there’s no key file to leak, copy or forget on a laptop. (For minisign it hashes the file locally and sends only the 64-byte hash, which is how a large binary gets signed by a key that can’t leave the service.) Here that service is AWS KMS, because that’s what the estate runs on, and a local key file works too for trying it out.

It’s a single binary that knows nothing about what language your project is written in, which is why Rust and other non-Go pipelines can sign with it without building anything of mine first.

Bring your own key store

AWS KMS is what I use, not what it’s tied to. The whole signing family was designed from the start so the key store is a plug-in. go/signing defines one small contract (take a key reference, hand back a standard Go signer) and knows nothing about any cloud: each key service is a module of its own, and the core can’t even compile in a vendor SDK, because a test fails the build if one sneaks in. AWS KMS is one such module and the local key file is another, and GCP KMS, Azure Key Vault, HashiCorp Vault or a PKCS#11 HSM would each be a third, written the same way. There’s a how-to for writing your own, and go/encryption follows the same pattern for decryption, down to a smartcard if that’s where your key lives.

Publish the key somewhere else

A signature is only as good as your confidence in the public key that checks it. The key is built into the tools that verify with it, and it’s also published over WKD (a well-known address on a domain I control, where gpg knows to look for it). The two copies have to agree, so forging a release means breaking into the forge and the key host at the same time. Publish your key where they can’t touch it goes through why that host was picked.

Verify before anything installs

go/signing is the library that checks those signatures. It’s what go-tool-base’s self-update uses: an update is verified against the built-in key and the WKD copy before it replaces anything, and a tool generated with signing switched on refuses an unsigned update outright. If the WKD copy can’t be reached at all, self-update carries on with the built-in key alone, so a tool behind a broken proxy can still patch itself, but two copies that disagree always stop it. krites is stricter about the models it downloads: there, an unreachable key host means try again later.

Encrypted reports, with no key to hold

go/encryption runs the same idea the other way round, for reports somebody sends you encrypted. The private half stays in KMS, and the only thing KMS is asked to do is the one secret step of the key exchange; everything after that, including the decryption itself, happens on your machine. So a person can read a report without the key ever existing anywhere it could be stolen from. Two encrypted emails in twenty years is where it came from.

The keys, as infrastructure

Two OpenTofu modules create the keys and the permissions around them. The signing module makes one KMS signing key and a role that only named CI jobs can use, through short-lived OIDC credentials. No person and no stored credential is granted signing, and a configuration that would let any job sign is refused before anything is created. The encryption module makes the decryption keys with no pipeline granted access to them, so reading a report is left to a person, and it refuses the key type that would break that, at plan time.

Decisions and what they cost

GPG over cosign. Signatures are checked with GPG and a key carried in the binary, chosen over cosign so verification still works offline and behind a firewall (go-tool-base spec 0056). WKD was picked to publish the key over Keybase, key servers and DNS. What it cost: there’s no transparency log, so if the forge and the key host were both broken into at once, nothing would notice.
A standalone signer. Signing moved out of go-tool-base into sigillum and a small module behind it (gtb spec 0176, sigillum spec 0001), because two projects were compiling the whole framework just to sign a file (cicd spec 0097). What it cost: signing now has two front doors, sigillum and gtb sign, over the same library.
Two keys where one would do. OpenPGP signatures use RSA and minisign uses Ed25519, because the OpenPGP library can’t sign Ed25519 with a key it can’t hold in memory, which is exactly the kind of key KMS gives you. What it cost: a project publishing both formats keeps two keys. One key or two is how that got settled.
Key stores as plug-ins. The libraries depend on a contract, never on a provider, and every key service lives in its own module (go/signing’s design notes), so adding one never touches the core and nobody compiles an SDK they don’t use. What it cost: sigillum picks its backends when it’s built, so a new key store means building sigillum with that module in it, and so far only AWS KMS and the local key file exist.
ECDH for decryption, not RSA. KMS’s RSA decryption only supports the padding OpenPGP doesn’t use, so reports are encrypted to a key-agreement key instead (sigillum spec 0004). What it cost: each certificate carries two keys, one to certify and one to decrypt.

Proof in use

Releases across the estate are signed with it: sigillum itself, colophon, krites and the artefact channel sign their checksum files with sigillum, rust-tool-base signs its binaries with minisign through KMS, and go-tool-base signs through its own wrapper over the same library.
Checked by hand on 10 October 2026: sigillum v0.5.2’s checksum signature comes back as a good signature in gpg, using the key fetched over WKD.
go/signing is a direct dependency of go-tool-base, keryx, krites, colophon and afmpeg, and reaches three more tools through go-tool-base. The key-service modules’ release merge requests run cicd’s go-core-currency check against the core’s latest release.
Eight tools build and publish their release binaries through cicd’s goreleaser component. colophon, krites and sigillum sign theirs with sigillum during that release, and go-tool-base signs through its own wrapper.
The security contact publishes its key the same way, over WKD: an RSA key to certify with and a P-256 key to decrypt to, the shape the encryption module builds.

Use it when, and when not to

Use it if you want people to be able to check that a release really came from you, using gpg or minisign they already have, and you’d rather the signing key lived somewhere it can’t be copied, whichever key store that is. It doesn’t care what your project is written in.

Know the limits first. Of the key stores, only AWS KMS and a local key file are built today. The others the design was made for are still outstanding, and until they land, anything else is a module you’d write yourself against the same contract. There’s no transparency log, timestamping or notarisation yet, and no key rotation policy, and a minisign key can’t be revoked. The encryption module runs on OpenTofu only.

Where it’s going

Key rotation, Linux package signing and Windows Authenticode are all drafted on go-tool-base’s wiki, alongside the Google Cloud and Azure KMS backends. sigillum’s own next step is exposing signing to AI agents as MCP tools (spec 0003).

The pipeline

Mon, 01 Jan 0001 00:00:00 +0000

The problem it exists for

One person can’t hand-maintain a CI file in 146 repositories. Every lint rule, security scan, release step and docs deploy would be written dozens of times and drift dozens of ways. So there’s one library of CI components instead, and 133 of those projects include it: 717 include lines between them, with the release component in 117 projects and the Go test, lint and security jobs in 100 each.

And the images those jobs run on used to be heavy. The lightweight jobs were pulling an image of nearly a gigabyte, 2.72 GB unpacked, carrying 2,582 scanner findings of every severity. Today they pull ci-base, under 80 MB.

How it works

Components, not copies

cicd is one GitLab project holding 38 CI components across nine tracks (Go, Rust, OpenTofu, docs, release and the rest), all released under one version stream. A project includes the components it needs and gets the whole job, rules and caching included. Each component has a self-test pipeline of its own, so a change is proven in cicd before it reaches anyone.

Hardened images, one per job

The jobs run on a family of small images, one per track, built from a Wolfi base and pinned to an exact digest. A job pulls the image for its language and nothing else. Build images covers how they’re built, scanned and kept current.

Runners that scale to nothing

The jobs run on a runner fleet of my own: one tiny manager instance always on, and spot-priced workers started on demand, from none up to four by default. Each worker takes one job, disappears after ten idle minutes, and is replaced after a hundred jobs, so a quiet evening costs next to nothing.

Who can fix it decides the gate

A check fails the pipeline only when the people running that pipeline can fix what it finds. The image scan is split in two for that reason, “because two different people can act on them”: findings in the operating system packages fail the build, since a rebuild clears them, while findings inside upstream tools are reported for their maintainers (image-scan). The skills gate works the same way. Hidden characters in a skill fail outright, and a suspicious-looking instruction is flagged for a person to read. Where a check is advisory, the component says why in its own header, because a job allowed to fail also hides a job that’s broken.

Security runs on every change

Most jobs skip a merge request that can’t have affected them, so a docs change doesn’t run the Go tests. The security jobs never skip, because “A dependency’s vulnerability status isn’t a function of the diff”, it’s a function of time (security, always on). A docs-only change merged the day after a vulnerability is published is exactly the one the scanner should see. The one exception came from measuring it: re-scanning colophon’s release merge requests, which change only the changelog, found nothing in 790 pipelines over three weeks and cost a fifth of the estate’s merge-request minutes.

Finishing what the tools start

Several components exist to carry one of the estate’s tools into every pipeline, so the tool does the work and the component makes it happen on time, everywhere:

colophon runs colophon’s release jobs in 118 projects, and goreleaser builds and publishes the binaries of the eight tools that ship them, go-tool-base and the tools built on it.
go-core-currency checks, on 98 projects’ release merge requests, that each family member is built against its core’s latest release, reading the families from this site’s own projects feed. That’s what keeps families like config, chat and git and forges in step.
skill-security is the security gate on the agent skills marketplace, and docs-verify runs a project’s own docs check, docscheck in krites’ case, on every merge request.
go-singleuse runs the controls analyzer for restarts that would reuse something they can’t, and tofu-module-publish publishes the AWS account foundation modules to GitLab’s module registry.

Decisions and what they cost

One library. Every component lives in one project (spec 0001). What it cost: every release reaches every consumer, which is a real load on the runners when it happens several times a week.
Track main, on purpose. Projects include the components from main instead of a pinned tag (spec 0079). With 106 repositories consuming cicd, 65 releases in 90 days had each been costing about a day of runner time in churn, and GitLab’s catalog and a stable branch were both turned down as the alternative. What it cost: it goes against GitLab’s own supply-chain advice, knowingly, and it puts the weight on cicd’s self-tests.

Proof in use

133 of the estate’s 146 active projects include cicd, and colophon releases 117 of them through it.
cicd is at v0.55, with 83 releases, and the images release on their own (Build images).

Use it when, and when not to

Use it as a model, or borrow components from it, if you run a lot of small GitLab projects on your own and want one place to change a pipeline.

It isn’t a general-purpose product, and the limitations page says so. It’s GitLab only, with no path to GitHub Actions, there’s no Python, PHP or Java track, the OpenTofu components sign in to AWS only, sites deploy to GitLab Pages only, and minor versions can still break things before 1.0. The runner fleet runs x86 jobs only, runs its workers privileged, and was built for one account.

Where it’s going

The open issues are mostly the fleet’s: build caches that don’t like being shared by concurrent jobs, and a GitHub mirror that keeps failing.

Transport stack

Mon, 01 Jan 0001 00:00:00 +0000

The problem it exists for

Every Go service needs the same handful of things: TLS that isn’t weak, timeouts, health probes, authentication, security headers, retries and circuit breaking, and logging and tracing on both the server and the clients it calls. Done piecemeal, each one is another chance to get a default wrong.

Defaults really are where it bites. Measured on this stack’s own server before it was fixed, a 13-second stream delivered 10 of its 13 frames, and the access log recorded a success, with a byte count higher than the client ever received (spec 0001).

How it works

One posture, everywhere

tls sets one TLS posture (1.2 at minimum, six chosen cipher suites) under the HTTP server, the gRPC server and both client factories, and transit is one middleware module that the server and the clients share. So the logging, tracing, retries and breakers on either end of a call come from the same place.

Clients and servers apart

transport is the server side (HTTP, gRPC and a REST gateway) on top of tls, authn and transit, and httpclient and grpcclient are the client side, in separate modules, so a program that only makes calls never links server code. The heavy extras are opt-in: an embedded API-docs UI, Prometheus metrics, and localca for trusted HTTPS on your own machine. Every one of the nine fails its build if a framework, a CLI library or a cloud SDK sneaks into its dependencies.

Each guard on the side it protects

Where a guard sits is part of the design. A rate limiter protects whatever is receiving the load, so it goes on the server. A circuit breaker protects a caller from a service that’s failing, so it goes on the client, and the framework offers neither the other way round, because “putting either on the wrong side is a category error” (transport middleware). On the client, the breaker sits outside retry. One call that retries three times and still fails counts as one failure, not three, so the breaker doesn’t trip too eagerly, and once it’s open, a call is refused before any time is spent retrying (transit’s middleware model).

Decisions and what they cost

Light clients, heavy server. The stack was split so client modules stay small, rather than one module dragging lifecycle, auth and gateway code into every program that only calls out (gtb spec 0123). What it cost: nine modules to keep in step, which grouped automated updates take care of.
A clean break. The server stack left go-tool-base as a clean break, with no facade re-exporting the old names (gtb spec 0129). What it cost: a breaking change downstream, eased by a symbol-by-symbol migration note.
Fix the logs, not a subsystem. The streaming problem above was fixed by repairing the logging middleware that hid failed writes, documenting how a stream outlives a timeout, and adding a small helper for server-sent events, instead of building a streaming subsystem (spec 0001). What it cost: safe streaming still depends on reading the docs.
The body cap stays on. Every route keeps a 1 MiB request-body cap by default, and a single handler can raise it for its own request (spec 0002), because dropping the server-wide cap would quietly undo a security-audit finding. What it cost: the cap is no longer a pure wrapper.

Proof in use

go-tool-base, keryx, krites and phpbotscout serve through it, and go/forge and its four adapters make their calls through httpclient.
75 releases across the nine modules since July 2026, with every consumer on the current one. Every module’s release merge request runs cicd’s go-core-currency check, which flags a release still built against an older transport core.
Nearly 600 test functions across the stack.

Use it when, and when not to

Use it if you’re writing Go services and clients and want hardened defaults that agree with each other, without adopting a whole web framework.

It reads no config files or environment variables and reloads nothing, renewed certificates included. There’s no HTTP/2 over plain text, no Unix sockets, no mutual TLS on the gRPC client, and httpclient doesn’t negotiate HTTP/2 or retry a POST. The breakers and rate limiters don’t coordinate across replicas, authn refuses shared-secret tokens and has no revocation, and localca’s Windows trust-store install doesn’t work yet.

Where it’s going

Making a server that dies after a clean start visible to its supervisor, and masking credential names in request logs, are the next fixes.

Web front ends

Mon, 01 Jan 0001 00:00:00 +0000

The problem it exists for

Three tools in the estate have a full web UI. keryx’s studio is where reels and social posts get made, krites’ studio is where a photo shoot gets culled, and Scout.DM’s portal is where a DM runs a campaign. Each one ships as a single Go binary, and the UI has to travel with it: no separate web server to install, no Node on the user’s machine, nothing to keep in step. Each one is also a front end onto a tool that already works from the command line, so the UI mustn’t become a second place where behaviour lives.

How it works

Svelte, not React or Angular

The choice was made for krites first, and written down there. Svelte “compiles to tiny vanilla JS, no virtual-DOM runtime”, which keeps the footprint and memory low and embeds cleanly in a Go binary, and the spec’s advice is to “Avoid a heavyweight React/Next stack for a localhost tool” (krites spec 0001). Vue was weighed and lost on footprint, and pure-Go native toolkits were turned down as an unfamiliar stack with a hand-built grid. The API in front of the UI doesn’t care which framework calls it, so “the choice stays reversible”.

keryx took the same stack from krites, and Scout.DM chose Svelte for its portal too. All three now run Svelte 5 on Vite 8, with Svelte’s runes for state rather than stores.

Plain Svelte first, SvelteKit where it earns it

keryx and krites are single-page apps in plain Svelte, with no router. keryx’s three screens are “conditional views”, and a router is to be reconsidered only “if state-sharing pain appears” (keryx spec 0011).

Scout.DM’s portal is bigger: more pages, two separate layouts, sign-in, and it’s reachable from outside the machine. So before choosing, the same four-route app was built both ways under the portal’s content security policy. Plain Svelte needed about 60 hand-written lines of routing, loading, focus handling and error pages for four routes, “and they shipped with a bug”: the old page stayed on screen showing the new page’s data. SvelteKit handled all of that itself and needed two hashes added to the server’s policy. So the portal is SvelteKit 3 with the static adapter, built to plain files with no server-side code, and a guard script refuses any SvelteKit file that would need a server to run.

Compiled into the binary

Each UI is built to static files and compiled into its Go binary. A placeholder page is committed in their place, so the binary still builds on a machine with no Node, and the placeholder tells whoever sees it to install a release. The release pipeline refuses to ship it. keryx found out why the hard way: without that guard, a build image missing Node “would build, sign, notarize and publish a release serving the ‘install a release for the full UI’ placeholder, entirely green”, and no Go test would notice, because they pass the same either way. cicd’s Svelte components build, lint, test and scan each UI, and the release waits for the build.

One API, typed at both ends

keryx and Scout.DM describe their API once, in OpenAPI, and generate both the Go server’s code and the browser’s TypeScript types from that one file. A check fails when the committed types fall behind it. krites documents its API by hand instead (studio API).

None of the three adds behaviour of its own. krites requires that the studio “exposes nothing the CLI/engine can’t do”, and keryx runs the same scenarios through the studio and the command line and checks they leave the same files behind. Estate architectural patterns covers that rule across the estate.

Locked down by where it listens

krites’ studio listens on the local machine only, and refuses any other address. keryx’s is open on loopback, and anywhere else it needs a one-time token that becomes an HttpOnly, SameSite=Strict cookie served over HTTPS (keryx spec 0018). It’s a cookie rather than a header because the browser’s image, audio and video requests can’t send a header. Scout.DM’s portal, which DMs reach from anywhere, signs people in with Discord and runs under a content security policy that allows only its own files, plus two inline items admitted by hash.

A small supply chain

keryx and krites ship no runtime npm dependencies at all. krites wrote its own photo-grid virtualiser rather than take a library, because “a compromised JS dependency ships as malware inside an Apple-notarized app” (krites spec 0014). Development dependencies are pinned to exact versions, and cicd’s security component checks every UI for known vulnerabilities, risky code and package provenance.

Tested in a real browser

Components are tested with Vitest and testing-library, and types with svelte-check. The two studios run Playwright against a mocked API, nine spec files each, and krites’ include a run with a 4,000-frame shoot to prove the grid only ever draws a small window of it. Scout.DM’s portal is driven by a headless browser against the real Go server, sending its real policy header, and the run fails on any policy violation, which is how every SvelteKit or Vite upgrade gets checked.

Decisions and what they cost

Svelte. The smallest runtime and the cleanest fit inside a Go binary. What it cost: where a library would have been the quick answer, as with krites’ photo grid, the estate writes its own to stay at zero runtime dependencies.
SvelteKit, measured first. What it cost: the portal builds to 172 KB against 56 KB for the plain version, and the server carries two hashes that break when SvelteKit moves a file, which it did in 3.0.
Generated, not committed. keryx builds its UI as part of Go’s generate step rather than committing the built files. What it cost: a release needs Node, and krites can’t be installed with go install, which skips the step and embeds the placeholder.
Types from one file. What it cost: the generated types cover the shapes of requests and responses, not the URLs, so a mistyped path is still a runtime error. Closing that would need a library at runtime, “and the SPA ships zero runtime dependencies”.

Proof in use

keryx, krites and Scout.DM, all on Svelte 5.57 and Vite 8, with Scout.DM’s portal on SvelteKit 3.
All three build, lint and test through cicd’s Svelte components, and keryx and krites run its security scan too.
Nine Playwright spec files in each studio, and a headless policy check on the portal.

Use it when, and when not to

Use it as a model if you ship a Go tool with a web UI and want it to stay one binary, one API and one set of behaviour. The placeholder-plus-release-guard trick works for any embedded front end, whatever the framework.

It’s built for UIs that sit in front of a tool, not for hosted web applications. All three are static builds with no server-side rendering, by design.

Where it’s going

New web UIs start on SvelteKit, carrying over the embedding, the typed API and the browser checks from all three, with plain Svelte still there for a tool small enough not to need it.