Go on PHP Boy Scout

I filed a feature request into my own framework

Fri, 03 Jul 2026 00:00:00 +0000

I’m building a tool called keryx, and the part of it that matters here is its studio: a browser app where the work happens, which saves everything you do into a git repository behind the scenes, the way a developer’s project lives in git with a history you can step back through.

I wanted that repository to be able to live entirely in memory. Cloned, edited, committed and pushed without ever writing a working copy out to a disk, for the times when you can’t, or would rather not, leave a checkout sitting around on the machine. It sounds exotic, but it’s something git libraries genuinely support, and it’s exactly what a browser studio running on a server somewhere wants.

Getting it working needed one small, awkward piece of plumbing in the middle. And a few lines into writing that piece, I stopped, because I realised I was writing it in the wrong repository.

The bridge I was about to vendor

Here’s the awkward bit. All of keryx’s file handling goes through afero, the standard filesystem interface in the Go world, the thing you hand your code so it neither knows nor cares whether it’s talking to a real disk, a test fake, or memory. It’s the interface go-tool-base hands you for filesystem work. But an in-memory git repository, the kind go-git gives you with its memfs, doesn’t speak afero. It speaks go-billy’s filesystem interface instead. Two perfectly good filesystem abstractions, and a worktree on the wrong side of the gap from all my code.

What I needed was an adapter: a bridge that makes a billy filesystem look like an afero.Fs, so the studio’s existing file handlers work unchanged over a repo that lives entirely in RAM. Twenty minutes of work, maybe. The obvious move was to write it inside keryx and get on with my afternoon.

And that’s the move I caught myself making. Because a billy-to-afero bridge is not a keryx thing. It’s not even a studio thing. It’s a general capability that any tool built on go-tool-base might want the moment it touches git. Vendor it in keryx and I’ve buried a reusable bit of plumbing inside one consumer, where it will drift away from the framework and get reinvented, slightly differently, in the next tool I build that needs it.

The bridge belonged in the framework. So that’s where I put it.

A feature request, against myself

I wrote the need up properly. Not a code comment, not a mental note, but an actual feature request, with a reference implementation sketched out, dropped into the go-tool-base repository as a document for the framework to act on.

There’s something slightly absurd about filing a feature request against your own project. The author and the customer are the same person. But that’s exactly what gives it its value. The most useful design input a framework gets is a real consumer hitting a real wall, and for once I was both: the person who maintains go-tool-base, and the person downstream of it who’d just discovered something it couldn’t yet do. The request wasn’t hypothetical or “wouldn’t it be nice”. It was “I am stuck on this right now, here is precisely what it can’t do yet.”

What came out the other side is pkg/vcs/repo/aferobilly, a first-class part of the framework as of v0.22.0. Its own description is the clearest summary of what it is:

// Package aferobilly adapts a go-billy/v5 Filesystem to an afero.Fs. It is the
// pure, reusable bridge behind pkg/vcs/repo's worktree-as-afero accessors, but
// works for any billy filesystem (memfs, osfs, chroot).

Alongside it, the worktree itself grew the accessors that hand you that view: WorkFS() for a live afero handle, and WithWorkFS() for an atomic sequence (worktree_fs.go, and the adapter itself). keryx then consumed it like any other framework feature, and the in-memory studio fell into place.

Two sessions, one dependency

The bit I’d actually recommend to anyone is what I did with my time while that got built.

I didn’t down tools and wait for the adapter. I handed the feature request to a separate agent session and let it build the framework feature from the spec, working in the go-tool-base repo, while my keryx session carried straight on with all the studio work that didn’t depend on the bridge. Two sessions running in parallel, deliberately sequenced around the one dependency between them: keryx needs the adapter, so the adapter session goes first, but only the last mile of keryx actually waits on it. When go-tool-base cut the release with the adapter in it, keryx pulled the new version and the final piece slotted in.

That’s a workflow the framework split makes possible. The thing that’s a shared capability gets built once, in its proper home, by one stream of work, while the thing that consumes it carries on in another. The dependency between them is real, so the order matters, but only at the very end.

The one rule that came with it

Upstreaming it also meant the tricky part got solved properly, once, with a warning attached, rather than learned the hard way in a consumer. The adapter is concurrency-safe by construction: it serialises every operation through a lock, so when that lock is the same mutex guarding the repo, a live afero handle over the worktree is genuinely safe to share. But that safety has a sharp edge, and the package says so plainly:

// A handle (and its open files) must NOT be used from inside a critical section
// that already holds the same locker (the repo mutex is non-reentrant — that
// would deadlock).

Use the handle inside a WithWorkFS callback and you’ll re-lock a non-reentrant mutex and hang yourself. That’s exactly the kind of footgun that, vendored in keryx, I’d have discovered at 11pm with a wedged process and no idea why. In the framework, it’s documented at the source, where the next consumer reads it before they trip over it.

The truest test of a framework

Building a real product on your own framework is the best test of it, and this is what that actually looks like in practice. The test is sharper than “does it work”. It’s “what does the product need that the framework doesn’t have yet”, and every real answer to that is a feature request waiting to be filed.

The discipline is filing it against the framework instead of patching around it in the app. Do that, and the awkward bridge has exactly one home, the deadlock warning gets written down once, and the next tool I build inherits all of it for free. The customer was me. The feature request was real. And go-tool-base is better for my having been stuck.

There's no AI in my photo culler

Wed, 01 Jul 2026 00:00:00 +0000

Before a wedding photographer can edit a single frame, there’s the cull: sitting down with three or four thousand photos from the day and deciding which are even worth keeping. The blurry ones, the ones where the flash fired into a mirror, the same moment shot eight times in a burst where only one frame is sharp. It’s mechanical, it’s exhausting, and it’s the first job krites does for Hailey.

Every culling tool I looked at before building it leads with the same word. AI. AI culling, AI selects, trained on millions of weddings. So when I sat down to write krites’ first pass, I assumed I’d be wiring up a model too. For the part that does the most work, it turns out, I didn’t need one.

The shipped culler doesn’t load a single weight. It’s arithmetic, the sort a calculator could do if you were patient enough, and that’s a deliberate choice rather than a corner I cut. Here’s what’s actually under it.

Blur is the variance of a Laplacian

The first question for any frame is whether it’s in focus. You can answer it without knowing anything about what’s in the photo.

A Laplacian is an edge detector. Run it over an image and it lights up wherever the brightness changes sharply, the crisp boundary between a dark suit and a white shirt, the line of an eyelash. A photo in focus is full of those sharp transitions; a soft or motion-blurred one has smeared them all into gentle gradients. So if you measure how much the edge response varies across the frame, a sharp photo gives you a big spread of values and a blurry one gives you a flat, lifeless number. That single number is the focus score.

In krites it’s a 3×3 kernel over the frame’s luma (the brightness channel, Rec. 601 weights), and the score is the variance of the response:

lap := int(luma[(y-1)*w+x]) + int(luma[(y+1)*w+x]) +
 int(luma[y*w+x-1]) + int(luma[y*w+x+1]) - lapCenter*c

Sum the responses, sum their squares, and the variance falls out as sumSq/n - mean*mean. No training data, no inference, and the same pixels always give the same answer. (quality.go.)

Exposure is a histogram

The second question is whether the exposure is salvageable. If a third of the frame is pure white, the highlights are blown and there’s no detail to bring back; if it’s mostly pure black, the shadows are crushed the same way.

That’s just counting. Walk the luma plane once, tally how many pixels sit at or above a near-white threshold and how many at or below a near-black one, divide by the total, and you’ve got two fractions: the blown-highlight proportion and the crushed-shadow proportion. A photographer cares about those two numbers directly, and a for loop produces them (quality.go).

Two photos are the same when sixty-four bits agree

Then there are the bursts. A photographer holds the shutter through the first kiss and gets twelve nearly-identical frames; you want the sharpest one and the rest out of the way. To do that the tool has to know which frames are “the same shot”, and again you don’t need to understand the photo to tell.

The trick is a perceptual hash, a difference hash to be exact. Shrink the image right down to a nine-by-eight grey thumbnail, then for each row note simply whether each cell is brighter than the one to its right. That’s sixty-four yes/no comparisons, packed into a sixty-four-bit number, a fingerprint of the picture’s broad light-and-dark structure that survives a resize, a small reframe or a touch of noise:

if grey[y*hashW+x] > grey[y*hashW+x+1] {
 h |= Hash(1) << bit
}

Two fingerprints are compared by counting the bits that differ between them, the Hamming distance, which on a 64-bit integer is one CPU instruction (bits.OnesCount64). A small distance means the frames look alike. krites only clusters consecutive frames within that distance, so a run of similar shots merges into a burst but two unrelated photos that happen to rhyme don’t (dedup.go).

Best-of-burst is then the dullest line of code in the project: keep the sharpest frame in the cluster, demote the others from keep to maybe, and write down why.

fv.Reasons = append(fv.Reasons, "near-duplicate of "+bestFrame+" (kept the sharper frame)")

Signals in, a verdict out

None of those measurements decide anything on their own. A focus score of 50 is rejectable on one shoot and fine on another, because the numbers scale with resolution and content. So the signals feed a profile, a small set of thresholds, and the profile turns them into a ruling: below the hard focus gate it’s a reject, below a softer floor it’s a maybe, blown past the exposure gates it’s a reject, otherwise it’s a keep. Every verdict carries its reasons in plain words, “out of focus (sharpness 32 below 50)”, because krites proposes and the human disposes (cull.go).

The seed thresholds for a wedding are just a starting point, written to config on krites init and tuned from there:

seedMinSharpness = 50 // below this: rejected as out of focus
seedSoftSharpness = 150 // below this (but >= min): demoted to maybe
seedMaxHighlights = 0.10
seedMaxShadows = 0.30
seedDedupDistance = 8

The thresholds are the whole point of keeping them visible. “Suitable for a wedding album” is Hailey’s definition, not mine and not a model’s, and a number in a config file is something she can move (profile.go).

Where the models do belong

I’m not claiming AI has no place in this. Some of what a wedding photographer culls on genuinely needs a model: is this person mid-blink, is anyone actually looking at the camera, is the composition any good. Those are coming, and they’ll be model-backed when they do. The deliberate bit is that they sit outside this deterministic core, behind an interface, opt-in. The maths that does the heavy lifting of the first pass never imports a model.

That separation buys three things you lose the moment a neural net touches the hot path. It’s reproducible: the same frames in the same order always cull the same way, so a verdict is debuggable and a regression is catchable. It’s quick enough to run over four thousand frames on a laptop with no GPU. And it stays honest about what it knows, because a threshold you can read is a threshold you can argue with, which a confidence score from a black box never quite is.

“AI culling” makes for a better headline. But blur really is just a number, a duplicate really is just sixty-four bits, and the grim, mechanical first pass that stands between a photographer and their best photos comes down to arithmetic.

A stack trace is not an error message

Tue, 30 Jun 2026 00:00:00 +0000

The repair agent I’ve been building into go-tool-base narrates what it’s doing as it goes. It builds, it tests, it lints, it fixes, and it logs each step so I can watch it think. Mostly that log is a calm, readable trickle: tried this, that failed, reading the file, here’s the fix. Mostly.

The moment a build or a lint step failed, the calm trickle turned into a wall of Go stack frames, the same forty lines of runtime gubbins over and over, burying the one line I actually wanted to read.

Two different things we both call “the error”

The agent’s tools wrap their failures with cockroachdb/errors, which is a lovely library: it attaches a stack trace to an error the moment you create it, so when something goes wrong deep in the weeds you can see exactly how you got there. A failed go build comes back as one of these rich, wrapped values, carrying its message and its stack.

The line that recorded the failure looked like this:

l.Warn("Tool execution failed", "tool", name, "error", err)

Looks fine. It is not fine. The logger is charmbracelet/log, and when you hand a structured logger a cockroachdb error value as a field, it renders the whole thing: message, wraps, types, and every frame of that attached stack. So every failed step, and during self-repair there are plenty, printed a full traceback at WARN. The signal, the actual build error, was in there somewhere, wearing a forty-line coat.

The thing is, a stack trace and an error message are two different objects that we lazily both call “the error”. err.Error() is the message: short, human, “lint issues found…”. The value err is the message plus the evidence of where it came from. They serve different readers. The message is for whoever’s watching the loop run. The stack is for whoever’s debugging why the loop itself is broken. Hand the wrong one to the wrong reader and you’ve got either noise or a mystery.

Pick a reader per level

The fix is to stop making one log line serve both:

l.Warn("Tool execution failed", "tool", name, "error", err.Error())
l.Debug("Tool execution failure detail", "tool", name, "error", err)

The message goes to WARN, where someone’s watching the agent work and just wants to know what failed. The full wrapped value, stack and all, goes to DEBUG, where someone’s gone looking for trouble and wants every frame. Turn the level up and the evidence is right there; leave it at the default and the loop reads like prose again.

It’s a one-line change, give or take, and it lives in the shared chat tool-dispatch path rather than in the agent, so every tool-using client gets the quieter log for free. It’s the same loop I’d just taught to respect the linter; apparently I was determined to make it both honest and readable in the same fortnight.

Where the stack belongs

The stack trace was never the thing I needed to hide. cockroachdb/errors attaching it is exactly what I want; it’s the whole reason I use the library. The mistake was where I let it surface. A trace dumped at WARN on every routine failure isn’t observability, it’s wallpaper, and wallpaper is what you stop seeing. Keep the loud version for the level where someone’s actually gone looking for it, and leave the everyday log alone. The stack was never the noise. Printing it on every line was.

Introducing afmpeg and ffmpeg-wasi: FFmpeg with no install, no CGO, no disk

Sun, 28 Jun 2026 00:00:00 +0000

keryx renders short promo reels, and the way it does that, today, is the way nearly everything does: it shells out to the ffmpeg binary. Which is fine, until you ask it to render a project that doesn’t exist on disk. keryx can work on an in-memory project, a repo cloned straight into RAM with no local checkout, and the moment it tries to hand that to ffmpeg, the whole thing falls over. The binary wants real files in a real directory. There aren’t any.

I went looking for a way out and didn’t find one I could live with. The bindings that use purego/dlopen are immature and still need the host’s libav libraries installed. The CGO bindings to libav are mature and can absolutely work in memory, but they’re CGO, and CGO takes away the thing I most want from a Go program: a clean static cross-compile to a single binary that runs anywhere. The famous ffmpeg.wasm is an emscripten build aimed at the browser, which is the opposite target from a server-side Go tool. And the one existing WASI-capable build pins FFmpeg 5.1, which is end-of-life, and I’m not shipping an out-of-support media decoder whose entire job is parsing untrusted files. Every road had a tollbooth I wasn’t willing to pay.

So I built two new things instead, and because they’re so closely tied I’m introducing them together.

ffmpeg-wasi: current FFmpeg, sandboxed, CGO-free

ffmpeg-wasi (repo) is the foundation, and it’s the harder of the two to build. It takes FFmpeg’s media libraries, the libav* family, and builds them to wasm32-wasi, then drives them with a small purpose-built engine, producing a single .wasm artifact that runs anywhere a WASI runtime does. No native FFmpeg install, no C toolchain at deploy time, no shelling out to a binary. It’s built to run under wazero, the zero-dependency pure-Go WebAssembly runtime, so a Go program can transcode, filter and mux media embedded, sandboxed, and CGO-free, still cross-compiling to one static binary.

The interesting bit, the reason this didn’t already exist, is a wall that FFmpeg 7.0 put up. The 7.x series rewrote the command-line tool to be mandatorily multithreaded, and a pure-Go WASI runtime can’t run that, because the threading model it needs (wasi-threads, spawning real threads) isn’t something wazero does. Every project trying to get current FFmpeg into WASI hits that wall, which is exactly why the existing build froze at the last single-threaded CLI, 5.1, and went EOL there. ffmpeg-wasi goes under the wall instead of over it: it doesn’t compile the CLI at all. It links the libav* libraries directly, which build single-threaded without complaint, and drives them with its own engine. That’s the move nobody else has made, and it’s the whole reason this can track current, maintained FFmpeg rather than a frozen one.

afmpeg: the pure-Go binding that lives in memory

afmpeg (repo) was the catalyst for all this, and it’s the part a Go developer actually touches. It’s a small, idiomatic Go API (New, Run, Probe, Close) sitting on top of the ffmpeg-wasi artifact, with one important twist: its I/O is bridged to an afero.Fs. afero isn’t in the standard library, but it’s the filesystem abstraction a great deal of Go already reaches for when it wants to swap a real disk for something else, and that “something else” is exactly the point here. The inputs and outputs of a media job can live entirely in memory, or in any afero backend you like, and ffmpeg is none the wiser. keryx gets to render its in-memory project without ever touching the disk, which was the whole reason I started.

This isn’t a roadmap post. Both projects are released and run today: afmpeg is at v0.4.0, ffmpeg-wasi at n8.1.2-1 (current FFmpeg, not the EOL 5.1), and between them they do real in-memory transcodes, verified end to end, WAV to AAC and H.264 rescaled and re-encoded with x264. Stripped of keryx’s reel-specific filtergraph, the shape a caller actually deals with is about this small:

// Compile the ffmpeg-wasi module once, then reuse the runtime.
rt, err := afmpeg.New(ctx, afmpeg.WithModuleFile("ffmpeg-wasi-lgpl.wasm"))
if err != nil {
 return err
}
defer rt.Close(ctx)

// The whole job lives in memory: no temp dir, nothing on disk.
fs := afero.NewMemMapFs()
afero.WriteFile(fs, "in.wav", input, 0o644)

// Drive it with the ffmpeg arguments you already know...
res, err := rt.Run(ctx, fs, "-i", "in.wav", "-c:a", "aac", "out.m4a")
if err != nil || res.ExitCode != 0 {
 return fmt.Errorf("render failed: %w (%s)", err, res.Stderr)
}

// ...then read the finished file straight back out of memory.
out, _ := afero.ReadFile(fs, "out.m4a")

The .wasm itself is a published release artifact you pin by SHA-256 (or let afmpeg fetch and verify for you with WithModuleURL), so the licence boundary stays explicit and nothing surprising ends up in your binary.

It doesn’t do everything yet, and I’d rather say so than let you find out the hard way. Single input to single output and Probe work now; the full multi-pad filter_complex and multi-output muxing are the next thing on the bench. But the part that nobody had cracked, getting current FFmpeg to run sandboxed, pure-Go, over a virtual filesystem with nothing on disk, is done, and you can pull it today.

Why it’s two repos

There’s a reason these are separate projects rather than one, and it’s about licences, not tidiness. The moment you compile FFmpeg you’re handling LGPL and GPL code, and I wanted that boundary to be obvious rather than smeared across one repo where nobody’s quite sure what’s covered by what. So the build tooling and the (L)GPL .wasm artifacts live in ffmpeg-wasi, with no grey areas, and afmpeg stays a clean permissive layer on top of the published artifact. I’m also shipping both LGPL and GPL builds of the artifact, so anyone who just wants the output and doesn’t fancy doing their own FFmpeg build can pick the licence that suits them and get on with it.

Both repos are public, so you can rebuild or relink either one yourself, and the doc sites are now linked in the nav up top.

There’s a stack of stories behind these two already, the threading wall and the spike that found the way under it, the afero-to-WASI filesystem bridge, the day I built a reel-shaped API and then reverted it before merge for being too narrow, the licence decision in full. I’ll be writing those up as the work lands. For now this is the headline: I needed an FFmpeg I could embed in a Go program with no install, no CGO, and no disk, couldn’t buy one off the shelf, so I built it in the open, and as of today it’s there to pull.

The agent said SUCCESS. The linter disagreed.

Fri, 26 Jun 2026 00:00:00 +0000

There’s a repair agent inside go-tool-base now. When you run gtb generate command, it doesn’t just spit out a file and wish you luck. An agent takes the generated code, builds it, runs the tests, and fixes whatever it broke, looping until the thing actually works (or until it’s tried the same fix five times and admits defeat). The whole point is that the generator hands you code that’s ready, not code that’s nearly ready and quietly now your problem.

So it stung a bit when I realised the agent had been holding itself to a lower bar than I’d hold any junior to. And I was the one who’d set the bar.

What “done” meant to the agent

The agent is a loop with real tools: it can build, test, read files, write files, tidy the module, and run golangci-lint. It works through them, and when it’s happy it replies with the word “SUCCESS” and the loop stops. On the Go side, the check is exactly that blunt:

if strings.Contains(strings.ToUpper(resp), "SUCCESS") {
 return nil
}

That’s the whole gate (agent.go). There’s no clever verification on my end that the agent actually did its homework. It does the work, it tells me it’s done, and I believe it. Which is fine, as long as the agent and I agree on what “done” means.

We didn’t.

The instruction that made lint optional

The agent decides it’s finished by following a numbered list in its system prompt. Here’s the line that did the damage:

If there are lint issues, use ‘golangci_lint’.

Read that the way the agent would. “If there are lint issues”… well, how would it know? The only way to find out is to run golangci-lint. But the instruction makes running golangci-lint the thing you do once you already know there are issues. It’s a chicken with no egg. And the SUCCESS condition at the bottom of the list never mentioned lint at all:

When the project builds successfully and tests pass, reply with “SUCCESS”.

So the agent did the sensible thing, given its orders. It built the code, ran the tests, saw both go green, and declared victory. golangci-lint was sat right there in its toolbox, unused, because nothing ever told it the job wasn’t finished until lint was clean too. I’d handed it a linter and then written a prompt that let it walk straight past it.

The galling part is that the linter was never the missing piece. The golangci_lint tool had been registered the whole time, and it even runs with --fix, so it’ll quietly clear the trivial stuff and only surface what actually needs a decision. The capability was there. The instructions just never required it.

The fix was words, not code

Here’s the part I find genuinely interesting. I didn’t add a check. There is no new gate in the Go. The fix is four lines of English:

Run ‘go_build’, ‘go_test’ and ‘golangci_lint’ in the project directory… Run all three; a clean build and passing tests do not imply clean lint.

Reply with “SUCCESS” only once ‘go_build’, ‘go_test’ AND ‘golangci_lint’ all pass with no errors and no reported issues.

That’s it. Lint moves from a remediation step you reach for once you somehow already know there’s a problem, into the gate itself. “Done” now means three green lights, not two.

It nags at me a little, that one. The reliability of an agent that writes and fixes real code came down to whether one sentence of instructions was precise enough. When your success criteria are a paragraph of prose, vagueness in that paragraph is a bug, the same as a vague type or an off-by-one. The spec just happens to be written in English, and the thing reading it is a language model that will cheerfully take the cheap reading if you leave it lying around. That’s the same lesson the goblin who wouldn’t stay dead taught me from the other direction: with these tools, what you say is what you get, and what you don’t say is fair game.

Leave it better, not just building

The Boy Scout Rule is the whole reason this blog exists, and I’d quietly exempted the robot from it. “Leave the campsite cleaner than you found it” had become “leave it building”, which is not the same thing and never was. If I’m going to put an agent in the loop precisely so it tidies up after the generator, then “tidy” has to mean what it would mean for a person on my team. Build, test and lint. No walking past the bin because nobody told you to pick it up.

The cobra hook I was sure I'd enabled

Wed, 24 Jun 2026 00:00:00 +0000

It came out of an audit. I’d recently pointed a small army of review agents at the whole go-tool-base codebase, back before that became a political problem, and one of the findings was that a subcommand could quietly skip the framework’s own start-up code. My first reaction was the dangerous one: surely not… we switched that on ages ago. So I asked for a second pair of eyes on the exact line.

There was no line. I was certain I’d enabled it, but I had simply never done it.

What the start-up hook is for

go-tool-base is built on cobra, the library most Go command-line tools are built on. In cobra, a command can carry a PersistentPreRunE: a function that runs before the command itself, and that, the name strongly implies, persists down to the command’s children. Think of it as the “before you do anything, get the tool ready” step.

go-tool-base uses exactly one of them, on the root command, to do all the humdrum setup: load and merge configuration, set up logging, ask about telemetry the first time you run, wire up the telemetry collector, and check whether there’s a newer release to install. Everything the tool does afterwards leans on that having happened. By the time your actual command runs, props.Config is meant to be populated and the collector is meant to exist.

The reasonable assumption (the one I’d made, anyway) is that “persistent” means it cascades. Define it once at the root and every mytool foo bar three levels down gets it for free.

“Persistent” promises less than it says

Here is the catch, and it is a good one to file away if you ever build a command tree. cobra runs only the nearest PersistentPreRunE it finds, walking up from the command you actually invoked. If a subcommand defines its own, that one runs and the root’s does not. Not as well as. Instead of. There’s no warning and no error; the child’s hook simply wins, and the parent’s is passed over in silence.

So the moment any command below the root declared its own PersistentPreRunE, the entire start-up for that branch, the config, the logging, the telemetry, the update check, would just not happen. props.Config would be nil. The collector would be nil. The first you’d hear of it is a nil-pointer panic a long way from the cause, or, worse, no panic at all and a tool running happily unconfigured.

EnableTraverseRunHooks is cobra’s opt-in to the behaviour most people assume is already the default: run every PersistentPreRunE from the root down to the leaf, in order. I’d assumed it was the default. It is not, and I’d never turned it on.

A landmine nobody had stepped on

The saving grace was that nothing was actually broken yet. In go-tool-base’s own command tree, the root is the only command that defines a persistent pre-run, so “root to leaf” and “nearest only” happen to produce the identical result. The flag being off changed nothing I could observe.

The bug was latent. It was a trap laid for the first person to do something entirely reasonable: add a PersistentPreRunE to one of their own subcommands. go-tool-base is a framework other tools are built on, so that person was never going to be me. The instant a downstream author did the obvious thing, their config and telemetry would vanish for that branch of their tool and nothing would tell them why.

That is the kind of bug I least like shipping. It compiled. It passed the tests. It would have demoed perfectly. And it sat there waiting to hand a stranger a debugging session with no breadcrumbs, for the crime of using a standard cobra feature the obvious way.

One line, and a note for whoever’s next

The fix is the line I was so sure already existed (root.go):

// Run every parent PersistentPreRunE from root to leaf rather than only the
// closest one. Without this, a downstream subcommand that defines its own
// PersistentPreRunE silently shadows the root bootstrap (config load,
// telemetry, update check) for that subtree. With it set, the framework
// bootstrap always runs first and a child hook runs after it.
cobra.EnableTraverseRunHooks = true

With it set, cobra runs the root start-up first and then the child’s hook, in order, so a downstream command adds to the setup instead of replacing it.

I didn’t want to stop there, because the next author to add a child hook still deserves to understand the ordering. So the change also drops a one-time debug line if it spots any command in the tree carrying its own PersistentPreRunE (the same file), saying out loud what’s going on:

l.Debug("a downstream command defines its own PersistentPreRunE; " +
	"it runs AFTER the framework bootstrap (config load, telemetry, update check), not instead of it")

And, belt and braces, the collector now defaults to a no-op so the few paths that do legitimately return early, like init and help, still satisfy the “always non-nil” promise the rest of the code relies on. The whole thing shipped with a pair of regression tests that assert the bootstrap really does run when a child hook is present, and that it runs first. It’s all written up in a short spec and landed in one commit.

Trust, but grep

There are two things worth taking away. The cobra one is portable: if you rely on PersistentPreRunE cascading down a command tree, set EnableTraverseRunHooks, because “persistent” means less than it sounds and the nearest hook wins by default.

The other is the one I keep having to relearn. The settings I’m most certain about are the ones I never check, precisely because the certainty is what stops me looking. Somewhere along the line I’d promoted “I meant to” straight to “I did”, with nothing in between… and then defended it out loud before I’d even gone to look. A review agent is good at exactly that blind spot: it has no memory of intending to do something, only the code in front of it. The best thing the audit turned up wasn’t a clever bug. It was a flag that was never there. Leaving the campsite better than you found it has to include the traps nobody’s stepped on yet.

I built my wife a judge

Mon, 22 Jun 2026 00:00:00 +0000

The morning after Hailey shot a wedding, we were both wrecked, lying in bed and not really moving, and she said she might just pay for Aftershoot to get through the cull. And my ears pricked up.

She’d been on her feet for the best part of fourteen hours the day before. She had four thousand photos waiting for her. And I, lying there next to her, realised that for the first time the thing I do in my spare time, the obsession that mostly just pays my half of the bills, might actually be of some use to her.

Four thousand photos and the weeks that follow

A bit of background. My wife is a photographer. Not full-time yet, but that’s the plan, and she’s been chipping away at it around life and the bills ever since she set up Echos Photography a couple of years ago. Her real love is landscape and animals, the patient stuff, but every so often she’ll take on a wedding or some portraits, and last weekend she was the photographer for a family friend’s big day. A lovely affair. She shot it; I flew the drone for a bit of aerial photography and video, which mostly means I got to stand in a field feeling useful while she did the actual work.

Then it ends, and the real job starts. Four thousand frames, and before a single one gets edited somebody has to sit and look at every one and decide which are even worth keeping. The blinks, the soft ones, the same moment fired off eight times in a burst where only one frame is sharp. Done by hand, in Lightroom, one at a time, that can take her weeks. Months, sometimes, depending on what else life is throwing at her. So when she said she’d pay a subscription to make the first pass go away, I understood completely. It’s a genuinely grim job.

I just thought I could do better… for her, specifically.

Why my ears pricked up

I’ve been building developer tools for a while now. go-tool-base has reached the point where it’s actually pleasant to build things on, and the work I’d been doing on keryx, a little studio you drive from a browser rather than a command line, had shown me the shape of something a non-developer could use without ever touching a terminal. The pieces were sitting there… they just needed pointing at a different problem.

So I built her a judge. That’s what the name means, as it happens. krites is Ancient Greek for “the judge”, and that’s the whole product in one word: the thing that looks at every frame and rules on it, keep, maybe, reject, with its reasons, so the human doesn’t have to do the first exhausting pass by hand. It judges; she still decides. It proposes, she disposes. The tool never gets the last word, and it was never meant to.

A tool that’s only trying to please one person

Here’s the thing an off-the-shelf product can never quite be: krites is opinionated entirely in Hailey’s favour. It’s built around her workflow, her kit, her idea of what makes a frame worth keeping, and it’s designed so that she can use it with no setup and no technical knowledge whatsoever. The day it asks her to edit a config file is the day I’ve failed.

I’ve also kept it deliberately dim. There’s very little machine learning in it, and that’s a choice rather than a limitation. Most of the first pass, deciding whether a photo is in focus or hopelessly blown out, whether it’s the eighth near-identical frame in a burst, turns out to be arithmetic, and arithmetic is simple, fast and predictable in a way a model never quite is. I’ll reach for ML where there’s genuinely no other way (telling whether someone’s mid-blink is the obvious one), and when I do it’ll lean on the Apple hardware she already owns, with a deliberate gap left so it can grow onto other machines later. But the rule is: no cleverness she doesn’t need.

And it all runs on her own machine, on purpose. No subscription, which matters when the photography isn’t paying for itself yet. No reliance on a connection, which matters because we’re hoping to spend a good chunk of the near future on the road (there’s a whole vanlife plan brewing, with a work-in-progress demo site slowly coming together), and a campsite in the middle of nowhere is exactly where you want to be culling a backlog and exactly where you won’t have signal. And nothing leaves the laptop, which matters most of all, because these are other people’s weddings. Their photos shouldn’t have to go and live on somebody’s cloud to be useful to the woman they hired.

Why not just pay for the thing that exists

Aftershoot is good. Genuinely good. The easy version of this post is the one where I sneer at the incumbent, and I won’t: they’ve built something clever, and plenty of photographers are well served by it.

But it’s a product, and a product is a compromise by definition. It’s built for the average of everyone, and it can only ever evolve in the directions a company decides are profitable. Hailey isn’t the average of everyone. Something I build is hers, and it can grow in any direction she can imagine, including the unprofitable, niche, only-makes-sense-for-one-photographer directions that no company would ever green-light. That’s not really a technical argument. It’s closer to pride. I’d rather build my wife the tool than rent her someone else’s.

Early days, and the people who made it possible

A word on where this actually is, though. What’s shipped today is the cull, the first pass I’ve described. The straightening, the cropping, the colour work, the little retouches that eat her evenings, the part where it learns her taste over time, that’s all roadmap, and I’ll write about each piece as it lands. krites is at the very start of its life.

None of it would exist at all without three things, and only one of them is mine. go-tool-base finally being solid enough to stand on. Being able to build, with a lot of AI assistance, far faster than I ever could have managed alone, fast enough that “I could probably build that” became “I have built that” inside a weekend. And Hailey, whose infinite patience and gracious kindness in indulging my passions is the only reason any of my obsessions ever get the room to become something.

Which is the part I keep coming back to. I’ve written before about the cost of the compulsion to build, the evenings it takes and the people it takes them from. krites is the same compulsion, for once, pointed the other way. The thing that usually pulls me away from her, aimed squarely at giving her her evenings back. I don’t think I’ve enjoyed building anything more.

She still gets the final say on every photo. Obviously. She’s the judge that matters. I just built her a faster one to argue with.

When you hand the same key to every call

Tue, 16 Jun 2026 00:00:00 +0000

I was building a tutorial, the kind where the whole point is that the reader runs every command and it just works. So I generated a fresh project with go-tool-base, added a command, then added a command underneath that command, and hit build. It didn’t.

pkg/cmd/hello/cmd.go: props.ChildCmd undefined
 (type *props.Props has no field or method ChildCmd)

My own generator, in my own framework, had just written code that referenced a thing that didn’t exist… which is a special kind of embarrassing.

A bug with a two-month alibi

git blame walked me straight to the commit that introduced the command middleware system back in March. Middleware here is the web-style idea of wrapping a command’s run function with cross-cutting behaviour, timing, auth, recovery, that sort of thing. To wire it in, the generator started emitting this for a nested command:

setup.AddCommandWithMiddleware(cmd, child.NewCmdChild(p), props.ChildCmd)

where the line before had simply been:

cmd.AddCommand(child.NewCmdChild(p))

The catch is that third argument. props.ChildCmd is meant to be one of a set of constants, but those constants are hand-declared for the framework’s built-in commands only (UpdateCmd, DocsCmd, and friends). The generator never declares one for a user’s child command, so the generated parent referenced a name that nothing had ever declared. Undefined. Won’t compile.

Here’s the part that should worry you more than the bug. It shipped in March and nobody noticed until late May. Partly because it only bites nested commands, a command under another user command; top-level commands register by a different path and were fine. But mostly because the generator’s tests checked the generated code as text, asserting that it contained the right strings, and never once ran go build on the result. CI was green for two months on code that could not compile. We were grading the essay without ever reading it aloud.

What the key actually was

Once I stopped staring at the missing name, the real problem came into focus… and it wasn’t the missing constant at all.

That third argument is a middleware lookup key. The framework keeps a table of middleware registered against each key, and the key tells it which to apply. It is not an on/off switch and it is not optional, so the generator had to supply one at every registration site. It was being asked to guess, on every call, a value it had no reliable way to produce for a user command.

And the tell was sitting right there in the same generator: everywhere else, the idiom was props.FeatureCmd("name"), a function that derives a key from a string. The nested-registration path was the one place that assumed a hand-declared constant instead. One call site out of step with all the others.

That is the actual lesson, and it has nothing to do with cobra or codegen. When you find yourself threading the same derived value through every single call site, and getting it wrong, the value is in the wrong place. The feature key was never the caller’s business. It belonged to the command.

Changing my mind about cobra

This is where I had to eat a helping of my own opinion.

go-tool-base is built on cobra, the de-facto Go library for building command trees, and I like it a great deal. I had deliberately not wrapped it. Every abstraction over a good library is a tax the reader pays, so my standing rule was: use cobra directly, don’t hide it behind something of mine.

The trouble is the middleware pattern kept growing, and the bigger it got the more plainly “don’t abstract cobra” was a position I was holding past its evidence. The very thing I’d refused to build was the thing the design had come to need. It helped that, having recently moved the project to GitLab, the version had reset to a 0.x prerelease, which makes a breaking change cheap. The window to stop patching and do it properly was open, and not for long.

Go’s composition model made it almost painless. You can embed a pointer to one struct inside another and the outer type inherits all the inner one’s methods for free, which is about as close to monkey-patching as a statically typed language gets. So a setup.Command became a cobra command plus the feature it belongs to:

type Command struct {
	*cobra.Command
	Feature props.FeatureCmd
}

func Wrap(feature props.FeatureCmd, cmd *cobra.Command) *Command {
	return &Command{Command: cmd, Feature: feature}
}

func (c *Command) Register(children ...*Command) {
	for _, child := range children {
		if child.Command == nil {
			continue
		}
		if child.RunE != nil {
			child.RunE = Chain(child.Feature, child.RunE)
		}
		c.AddCommand(child.Command)
	}
}

Because *cobra.Command is embedded, a setup.Command is a cobra command for every method cobra offers; the one place you need the raw pointer, you reach for .Command. The generated command now carries its own identity:

func NewCmdChild(props *props.Props) *setup.Command {
	cmd := &cobra.Command{Use: "child", RunE: ...}
	return setup.Wrap(props.FeatureCmd("child"), cmd)
}

and the parent registers it with nothing threaded through the call:

parent.Register(child.NewCmdChild(p))

The feature key now lives on the command, derived from the command’s own name, which is the one place the generator can always produce it correctly. The bug isn’t so much fixed as made unsayable: there’s no call site left to write the wrong thing into. The wiring got cleaner on the way past, too, each command’s run is wrapped exactly once with its own feature, instead of the old recursive pass that re-applied the parent’s feature down the whole subtree. And the old free function stays on as a deprecated shim that just calls Register, so nothing downstream breaks before v1.0.

Changing my mind about the tests

The redesign was the satisfying fix. The test was the important one.

The reason a non-compiling generator sailed through CI for two months is that its tests read the generated source as text. A generator is a program that writes a program, and we were checking that it wrote the expected words without ever asking whether the words formed a working program. So the redesign shipped with a different kind of test, one that scaffolds a real project, adds a nested command, and actually builds it. Its own comment says the quiet part out loud:

The previous test suite asserted file-content shapes but never tried to go build the generated module, so the nested-command path that referenced undefined props.<Name>Cmd symbols compiled cleanly in tests and broke only when downstream users built their tools.

It’s gated behind an integration flag, because it shells out to the Go toolchain and that’s too heavy for every unit run, but it closes the exact gap that hid the bug. The only real test of a code generator is whether its output compiles.

What it comes down to

Three times in one bug I had to change my mind. I’d decided cobra shouldn’t be abstracted; the evidence said abstract it. I’d reached for the one-line patch; the evidence said redesign. I’d trusted tests that read the generated code; the evidence said build it. None of those were comfortable, and the version reset is the only reason the timing was kind.

Both technical lessons are worth keeping. When the same derived value is threaded through every call, the abstraction is in the wrong place. And the only proof that something which writes code actually works is to compile what it writes. But the one underneath both, the one I apparently have to keep relearning, is simpler than either: don’t get so attached to an implementation that you can’t change your mind when the evidence says it doesn’t fit. The framework is better for it, and it now has a composition seam to hang the features cobra doesn’t give us natively. A nested command that wouldn’t build was just the thing that finally made me look.

Sign your own binaries with go-tool-base, part 1: sign and verify on your laptop

Mon, 15 Jun 2026 00:00:00 +0000

The quickest way to understand release signing is to do it once, by hand, with nothing but a key on disk. No cloud account, no CI, no cost. This first part of the signing series walks the whole loop on your laptop: make a key, sign a file, and verify the signature, including with plain gpg so you can see it isn’t locked to anything of ours. Everything you learn here maps one-to-one onto the AWS KMS workflow in the later parts; only where the key lives changes.

You’ll need the gtb CLI (installation docs). Make a scratch directory to work in, because we’re going to create a few files.

Make a signing key

gtb keys generate creates a keypair entirely inside the process and writes both halves to disk:

gtb keys generate \
 --algorithm rsa \
 --name "Acme Releases" \
 --email "release@acme.dev" \
 --created "2026-06-01T00:00:00Z" \
 --output signing.asc

INFO Generated OpenPGP keypair algorithm=rsa public_output=signing.asc private_output=signing.pem creation_time=2026-06-01T00:00:00Z fingerprint=...
WARN Move the private-half file to offline storage now. private_output=signing.pem

Two files come out: signing.asc (the public half) and signing.pem (the private half, a PKCS#1 PEM). The private half is the thing you guard. There’s no on-disk passphrase in this version of gtb, so keep it under filesystem encryption (LUKS, FileVault, or wrap it with age) rather than leaving it lying about.

One flag is doing quiet but important work: --created. An OpenPGP key’s fingerprint is derived partly from its creation time, so if you let it default to “now”, every run produces a different fingerprint. Pin it to a fixed instant and the key is reproducible, which matters the moment you start embedding it in a binary. Get in the habit now.

Mint the public key you’ll actually publish

You could hand signing.asc around as-is, but we’re going to produce the public key a slightly different way, with gtb keys mint:

gtb keys mint \
 --backend local \
 --key-id signing.pem \
 --name "Acme Releases" \
 --email "release@acme.dev" \
 --created "2026-06-01T00:00:00Z" \
 --output release.asc

INFO Minted OpenPGP key backend=local key_id=signing.pem output=release.asc creation_time=2026-06-01T00:00:00Z fingerprint=...

mint wraps a signing backend in OpenPGP framing and writes out the armored public key. Here the backend is local (a PEM file on disk), but in production it’ll be aws-kms pointing at a key you can’t hold. Minting the public key from the backend is the one habit worth forming early: it’s the only way to get the public half of a KMS key, so doing it the same way locally means the rest of the series is identical bar one flag. release.asc is the key you publish and embed from here on. (Because we pinned the same --created, its fingerprint matches the generated one exactly.)

Sign something

A real release signs its checksums.txt, so make a stand-in and sign it:

printf 'abc123 acme_linux_amd64\ndef456 acme_darwin_arm64\n' > checksums.txt

gtb sign \
 --backend local \
 --key-id signing.pem \
 --public-key release.asc \
 checksums.txt

INFO Signed file backend=local key_id=signing.pem public_key=release.asc input=checksums.txt output=checksums.txt.sig ...

That writes checksums.txt.sig, a detached, ASCII-armored OpenPGP signature. Note gtb sign takes --public-key: it cross-checks that the backend key matches the public key you claim to be signing as, and refuses if they diverge, so you can’t accidentally sign with the wrong key.

Verify it, two ways

First, the way your tool will do it on every self-update: against the public key. That path is the subject of a signature the platform can’t forge and we wire it into a real binary in part 5. For now, prove the signature is sound with something every machine already has, gpg:

gpg --import release.asc
gpg --verify checksums.txt.sig checksums.txt

gpg: Signature made ...
gpg: using RSA key ...
gpg: Good signature from "Acme Releases <release@acme.dev>"

Good signature is the whole point. The signature gtb sign produced is an ordinary OpenPGP detached signature, so anyone can verify it with the standard tool, no go-tool-base required. (gpg will warn the key isn’t certified in its web of trust; that’s expected and unrelated to whether the signature is valid.)

Now change a byte of checksums.txt and run the verify again. gpg reports BAD signature. That failure is the entire reason any of this exists: a tampered manifest no longer matches the signature, and a tool that requires a valid signature will refuse the update.

Where this leaves you

You’ve signed a file with a key you made and verified it independently. That’s the complete trust loop in miniature, and the shape never changes: a private key signs, a public key verifies, and the two are produced and checked the same way whether the private half is a .pem on your laptop or an HSM-held key in AWS.

The local key was the easy bit, and also the weakest: it’s a file, and files get copied. Part 2 moves the private key somewhere it can’t be copied at all, AWS KMS, and the only command that changes is the --backend flag.

Everyone wants Rust's safety, nobody wants Rust

Sun, 14 Jun 2026 00:00:00 +0000

This spring, the better part of a million lines of Zig quietly became a million lines of Rust. Bun, the JavaScript runtime that was the showcase for “you don’t need a borrow checker, you need good tools and a steady hand”, looked at its own memory bugs and switched teams. Around 99.8% of its test suite passed on the rewritten code, a clutch of memory leaks closed in the move, and the maintainers said the quiet part out loud: the previous release would be the last one written in Zig.

It’s tempting to read that as Rust winning, hoist the flag, and move on. I don’t think that’s quite the story, and the more interesting one is happening everywhere else at the same time.

Because Bun is the exception that went all the way. Everyone else is trying to get the safety without the Rust, and watching how they’re going about it tells you more than one runtime’s heroic rewrite does.

What everyone’s actually after

A quick level-set, because not everyone reading this writes systems code daily. “Memory safety” is the property that a program can’t read or write memory it has no business touching: no using a value after you’ve freed it, no running off the end of an array. It sounds niche. It is, by most counts, behind something like 70% of serious security vulnerabilities, which is why governments and trillion-dollar companies suddenly care a great deal.

There are roughly three ways to get it. Rust uses a borrow checker: a compiler that flatly refuses to build your program unless it can prove, before it ever runs, that you never touch memory after you’re done with it. The price is that it argues with you the entire time you’re writing. The product is that an entire category of bug becomes literally unwriteable. Go (and most managed languages) uses a garbage collector: a runtime janitor that frees memory for you, so you mostly can’t get it wrong, at the cost of some overhead and a little control. And then there’s the old way, the one most code on Earth still uses: trust the developer to get it right, and add an escape hatch, usually a keyword like unsafe, for the bits where they promise they have.

The retrofit trend is everyone in that third camp trying to inch toward the first two without rewriting the world.

Rust didn’t invent any of this

Worth saying plainly, because the fan club rarely does: Rust invented almost none of it. The borrow checker is, by Rust’s own admission, Cyclone’s region-based memory management, from a safe-C experiment in the early 2000s, welded to affine types out of linear logic, ideas that predate Rust by decades. And it goes beyond the borrow checker. Rust’s exhaustive pattern matching came from ML and Haskell. Its “errors are values, and there is no null” approach, Result and Option, is Haskell’s Maybe and Either in work boots.

What Rust did, and did better than anyone before it, was taste and integration: it curated thirty-odd years of academic research into one coherent language and proved the ideas could carry real systems code rather than just research papers. That is the genuine USP, and it’s why the rest of the industry is now shopping from the same shelf. Pattern matching has landed in Python and Java, with a proposal in flight for C++26. Swift 6 shipped compile-time data-race safety, its Sendable machinery a close cousin of Rust’s Send and Sync. The borrow checker just gets the headlines because it’s the hardest bit to copy. Which makes the title almost too literal: everyone wants Rust’s safety, and they are quietly adopting its mechanisms one feature at a time.

Credit where it’s due: C# is doing this properly

The example that made me sit up is C#. In C# 16, Microsoft is redefining the unsafe keyword that’s been in the language since version one. Instead of unsafe marking a lump of syntax, it now marks a contract: a promise the compiler can’t verify and a human has to read and uphold, with documentation and static analysers nudging you to take it seriously. They’re even floating badges on NuGet packages to show which ones have opted in.

My first instinct with any retrofit is suspicion, because bolting safety onto a language after the fact has a long and miserable history, and an escape hatch that’s easy to reach is an escape hatch people will reach for the moment they’re in a hurry. But this isn’t a bolt-on. Taking the keyword that’s already there and giving it real teeth is working with the grain of the language instead of stapling a second safety system alongside the first. That’s honest engineering, and it deserves the credit. It genuinely raises the floor.

A contract is not a guarantee

Here’s where my enthusiasm meets its limit, and it’s a distinction I happen to have a lot of skin in.

C#’s redefined unsafe makes dangerous code visible and reviewable. That is a real improvement, and most teams would be better off for it. But visible and reviewable still means a human has to honour the promise. It’s a sign on the door. Rust’s equivalent is a wall: in rust-tool-base I put #![forbid(unsafe_code)] at the top of all eleven shipping crates, and forbid is not advice, it’s a refusal. The compiler will not build a crate that contains unsafe, full stop, and unlike its softer sibling deny, you can’t quietly switch it back off in a corner of the code where it’s inconvenient. The whole reason I use forbid and not deny is that I don’t trust future-me, in a hurry, not to reach for the hatch.

So when I look at the C# work I think: good, genuinely good, and they should take it further. A contract a human upholds is not the same kind of thing as a proof a compiler enforces, and the trend, if it’s serious, points at enforcement. Visible is better than invisible. Impossible is better than visible.

Discipline never scaled, and that’s not an insult

The objection I keep hearing, and that a younger me would have made, is that any language can be memory-safe if you’re just disciplined enough. And it’s true, in the way that any house can be tidy if you never get busy. In the before times we shipped memory-safe C with code review and valgrind and sheer bloody-mindedness, and it worked, sort of, at small scale.

It doesn’t scale, and Bun is the proof sitting on the table. That wasn’t a sloppy team learning the basics. It was a strong team, betting publicly on the discipline-and-good-tools model, and the memory bugs piled up anyway until the honest move was to let a compiler take the job. Discipline failing at scale isn’t a moral failure of the engineers. It’s just what happens when you ask humans to hold a thousand invariants in their heads across a million lines. Delegating that to a machine that never gets tired or rushed isn’t laziness. It’s the entire point of having compilers at all.

The part that changed my mind

I learned most of my Rust by building rust-tool-base with an AI alongside me, leaning on it to explain the borrow checker, suggest the idiomatic shape, and check my work. And somewhere in that I noticed the thing I now can’t unsee: the borrow checker is exactly as good a guardrail for the AI as it is for me.

A model, like a tired human, will write a confident use-after-free without blinking. In Rust it simply doesn’t compile, so the mistake never reaches me. What that does is move the whole error surface. The bugs that survive into review aren’t memory bugs or lifetime bugs or data races, the language has eaten those, they’re errors of logic: the code is safe and wrong. And logic is precisely where I want my attention, and the AI’s, because it’s the part a human has to own and the part the models are getting better at every month. (I split my AI work across a few providers for their different strengths, so this is not a pitch for anyone’s logo. The effect is the same whoever’s doing the typing.)

Which dissolves the one argument that ever really kept people out of Rust. “The borrow checker is too much friction” was always the case for the defence. But Bun’s million-line rewrite was done largely with an AI, because an AI is very good at paying a tax that is tedious and mechanical rather than creative. The friction is getting cheaper to pay at exactly the moment the guarantee is getting more valuable to have. In an AI-assisted world, a language that proves safety is worth more, not less, because it fences in the machine’s mistakes as firmly as your own.

None of this means rewrite everything in Rust

I want to be careful not to land somewhere smug, because most software does not need what Rust offers and pretending otherwise is how you end up rewriting a CRUD app nobody asked you to. Garbage collection is not a failure state. Go’s collector keeps getting meaningfully better, my own go-tool-base is GC’d top to bottom and I have never once wished it weren’t, and “safe-by-default with a GC” is the right answer for a vast amount of the work most of us do. The borrow checker is a price, and you should only pay it when the thing you’re buying, that last class of guarantee with no runtime cost, is something your stakes actually need.

What it comes down to

The question was never “is it as safe as Rust”. That framing turns everything into a loss for everyone who isn’t Rust, which is silly. The useful question is: what does your language make the default, and how hard does it make the escape hatch to reach? Go makes safety the default and charges you a GC. Rust makes it the default and charges you the borrow checker. C# is moving its default in the right direction and, for now, leaves the hatch as a promise rather than a wall.

Credit the retrofits, they are raising the floor for an enormous amount of code that was never going to be rewritten. Just don’t mistake the floor for the ceiling, or a contract a human signs for a guarantee a compiler keeps. Everyone wants Rust’s safety, and the interesting question, now that an AI will pay the toll for you, is who still has a reason not to want it.

Widen the lens past Rust, though, because that’s where the news gets genuinely good. We’re at a turn in how languages evolve. Compile-time rigour is spreading rather than retreating: borrow checking is reaching the Python family through Mojo, static typing long since conquered JavaScript, and even the managed languages are turning their escape hatches into something you have to argue with. More of our safety is quietly moving from “remember to” into “can’t not”. And the one thing that always made the strict path hard to start down, the friction, is being absorbed by an AI that will happily learn the rules so you can lean on them. I’ve been at this long enough to distrust a rosy forecast, but I’ll put my name to this one: the outlook for software that’s safe and secure by default has never looked better.

Sign your own binaries with go-tool-base

Sat, 13 Jun 2026 00:00:00 +0000

If your CLI tool can update itself, it has a decision to make that nobody is watching: when it pulls down a new version, should it trust what just landed? A checksum tells it the bytes match a manifest. It does not tell it who wrote the manifest. Close that gap and your users get updates they can actually trust; leave it open and a compromised release host can hand them anything it likes. This series is the end-to-end “how”, using the signing tooling built into go-tool-base.

By the end you’ll have a CLI that ships releases signed by a key you control, verifies its own updates against that key, and does the whole thing with no gpg wrangling and no long-lived secrets sitting in CI. We did the why and the how it works in two deep-dives already, a signature the platform can’t forge and a signing key that never leaves KMS. This is the use-it counterpart.

What you’re protecting against

Nobody’s coming to clean your supply chain, so it’s worth being clear about the threat before you spend an afternoon on the fix. A checksum file sits next to the binary on the same release page. Whoever can swap the binary can swap the checksum in the same breath, and the hash still matches. A signature is different: it’s made by a private key the release platform never holds, and verified against a public key your tool fetches from somewhere the platform can’t reach. To forge a release that passes, an attacker would have to steal a key that, done right, was never anywhere they could get at it.

That “done right” is the whole series.

Two paths through it

You don’t need a cloud account to start. The series runs in two stages:

Learn it locally. Part 1 signs and verifies on your laptop with a plain key on disk. No AWS, no CI, no cost. It’s the fastest way to see every moving part for real.
Do it for production. Parts 2 onward move the private key into AWS KMS, where it’s generated and never leaves, and wire your release pipeline to sign through it over short-lived OIDC credentials.

Each part stands on its own and ends with something that works. They build in order, but you can stop after Part 1 with a genuinely useful skill and come back for the cloud parts when you need them.

Before you start

You’ll want a CLI built on go-tool-base to sign. If you haven’t got one, the Building a CLI with go-tool-base series gets you there in an afternoon; this one picks up where releases come in. You’ll also need the gtb CLI installed (the installation docs have the one-liner), and for the cloud parts, an AWS account and a GitLab or GitHub project to release from.

The parts

Sign and verify on your laptop: gtb keys generate, gtb sign, and gpg --verify, the whole loop with a local key.
A signing key in AWS KMS: stand up an asymmetric KMS key with the terraform-aws-signing-kms module.
Keyless CI signing with OIDC: federate GitLab and GitHub into the signer role, no stored credentials.
Mint and publish your public key: gtb keys mint from KMS, then gtb keys wkd to publish it off-platform.
Embed the key and require verification: bake the trust anchor into your binary and turn enforcement on safely.
Sign every release with GoReleaser: wire signing into a real tagged-release pipeline.
Rotation and break-glass: the part everyone skips, and how to do it without locking anyone out.

Start with Part 1. By the time you reach the end, the chain runs from a key born in a vault to a binary on a stranger’s machine checking, on its own, that the update it just fetched is really yours.

The goblin that wouldn't stay dead

Fri, 12 Jun 2026 00:00:00 +0000

Turn one, the player swings, the die comes up 20, and my AI dungeon master narrates the goblin falling silent, leaving the player alone in the corridor. Good. Turn two, another roll, a 6 this time, and the same dungeon master cheerily has the goblin “dance back” out of the dark to take another swing. The goblin I’d just watched die was up and fighting again, and the model didn’t so much as blink.

I didn’t feel cheated, or even surprised. I felt the small, familiar thud of oh, yeah, I forgot that bit. Because the model hadn’t gone rogue. It had done exactly what a language model does. The gap was mine.

This was the war story behind part four of the go-tool-base tutorial, the AI dungeon master. The tutorial shows the clean, final design and quietly moves on. It doesn’t show the three different ways I got it wrong first, which is a shame, because the wrong turns are where the actual lesson is.

Why a dungeon master at all

A word on why I was even here. I was trying to prove the chat component of the framework to myself. There’s a voice that pipes up whenever I build anything in this space, “LangChain exists, who do you think you are?”, and the answer I keep landing on is that LangChain is enormous and I wanted something small enough to hold in your head. The tutorial was the test: could a newcomer wire AI into a CLI with it and come out the other side with something that actually behaves?

That last word is the whole problem. A tutorial has to leave you holding something dependable, and dependability is the one thing AI fights you on. I also wanted it to be fun, a thing someone might keep poking at after the tutorial ends, maybe even the hook that gets a person other than me to use the framework. I batted hook ideas around and liked none of them, until the obvious one landed: I run a tabletop game on the odd weekend, so make the AI the dungeon master. Gamify the thing. Then watch it raise the dead.

Strike one: nothing to enforce

The first version was the naive one. I gave the model a roll tool, because the one thing you absolutely cannot let a language model do is pick its own numbers, and otherwise let it narrate freely. The conversation history carried from turn to turn, so it remembered the fight. I assumed remembering was enough.

It isn’t. Remembering and being held to it are different things. The history told the model a goblin had died; nothing stopped it writing the goblin back in when the next turn’s narration wanted a bit of jeopardy. Memory is not a constraint. The model will happily contradict its own past if you’ve given it room to, and I had given it nothing but room.

Strike two: a tool to read the state

The obvious fix, and I do mean obvious, the kind you reach for without thinking, was to give the model a state tool so it could check who was alive before it narrated. Hand it the facts on request and surely it’ll stop making them up.

What it actually did was dither. Handed a tool it could call to look things up, it called it. And called it. And called it again, turning a turn over in its hands without ever committing to an action, burning through its step budget on lookups and leaving the player staring at nothing. I’d cured the lying by inventing paralysis. A tool the model can call is a tool it will call, often instead of doing the thing you actually wanted.

Strike three: refereeing its own dice

When I did get it reading state cleanly, the third failure crept in, and this one was subtler. Once the model could see the goblin’s hit points, it started deciding the fight. It would read that the goblin had 12 HP and just narrate a killing blow, hits and damage and all, without calling the roll or attack tools at all. Why ask the dice when you can see the board and write whatever outcome the story wants? Give a model enough context and it stops being a narrator and starts being a referee, which is precisely the job I’d built tools to keep out of its hands.

The fix was less, not more

Three failures, and notice the shape of my fixes: each one added something. More memory, then a tool, then more context. Every instinct said the model needed more to work with. Every time, the extra capability was the new way to be wrong.

So I went the other way. The truth lives in a plain Go struct that I own, not the model. There’s no state tool to dither on, because the loop simply prepends the current state to every turn’s input, fresh, so the model never has to ask and never gets to drift. The mechanics, the dice and the damage, live in Go functions the model has to call, and the system prompt says in as many words that it must not decide a hit or damage itself. The model is left with exactly one job: narrate. The prose is its to invent. The maths, the state and the shape of the result are not.

That’s the line that turned three bugs into a feature. You don’t make a language model reliable by giving it more to work with. You make it reliable by giving it less to be wrong about.

The freedom I chose not to give it

There’s a real tension in that, and I want to name it rather than pretend the boxed-in version is the only true one. At my own table the rules are guidelines, not guardrails. I ignore them, bend them, improvise, reach for the “rule of cool” when the moment’s better for it. A great AI dungeon master would have that same freedom, and a few out there genuinely do, Old Greg’s Tavern is a lovely example of how far the free-form version can go.

But that freedom costs far more than a tutorial can spend, and it buys unpredictability I was specifically trying to teach people to avoid. So I made a deliberate trade: guardrails instead of guidelines. Simple, but not so simple it’s boring. The player still gets a “not on rails” game, they can try anything and the DM copes, but every outcome that matters runs through code I trust. That’s the right shape for a tutorial, and, not by coincidence, the right shape for most AI features you’d actually ship.

What the goblin taught me

The thing I keep coming back to is that the model never misbehaved. It resurrected the goblin because I gave it the freedom to. It dithered because I gave it a button to press. It refereed because I let it see the board. Every failure was a permission I’d handed over without meaning to. The reliability didn’t come from a cleverer prompt or a bigger model, it came from working out, one dead goblin at a time, exactly how little the model needed to be trusted with.

If you want the version where it all works first time, the tutorial has it, the tool-calling and the typed turns wired up properly. This was the road there. The goblin, you’ll be glad to hear, now stays down.

A signing key that never leaves KMS

Thu, 11 Jun 2026 00:00:00 +0000

The last post in this series walked through how a tool verifies a release signature the platform can’t forge. That post had a loose end dangling off the back of it, and I knew it the whole time I was writing. Because a signature has to be produced by a private key… and a private signing key is the single worst thing in this entire story to lose. Steal it, and you sign malware that sails through every check I spent two posts building, signature and all. So where does that key live? The answer I landed on is the one this whole post is about: inside AWS KMS, and it never comes out.

The only key you can’t steal

Think about where a signing key normally ends up. A file on a build server. A secret in CI. A key on the release engineer’s laptop, “just for the release, I’ll delete it after”. Every one of those is a copy, and every copy is one more thing somebody can read, exfiltrate, or quietly clone while your back is turned. You can wrap them in passphrases and vaults and rotation policies all you like, and you’re still standing guard over a thing that exists in a place you don’t fully control.

The way out is almost annoyingly simple to state: the only key nobody can steal is the one that was never anywhere to be stolen from. So don’t hold the key at all. Let something else hold it, somewhere it has no export path, and ask that thing to sign for you.

That thing is AWS KMS. This is the infrastructure side of the question I opened the signing series with, finally answered with real Terraform.

A key that’s born in the box and stays there

The signing key is an asymmetric KMS key, and the module that provisions it is small enough to read in one sitting:

resource "aws_kms_key" "this" {
 description = var.description
 key_usage = "SIGN_VERIFY"
 customer_master_key_spec = var.key_spec # default RSA_4096

 # Asymmetric SIGN_VERIFY keys do not support KMS-managed rotation;
 # rotation is handled by minting a new key (alias = `<name>-v2`) and
 # publishing the v2 public key alongside the v1 key (dual-sign window).
 enable_key_rotation = false

 policy = data.aws_iam_policy_document.key_policy.json
}

The private half of that key is generated inside KMS and there is no API that hands it back to you. You don’t sign with it the way you’d sign with a file. You call kms:Sign: the bytes you want signed go up, a signature comes back down, and the key itself never moves. An attacker who completely owns my CI, my account, my laptop, can ask KMS to sign things for as long as their access lasts… but they can’t walk off with the key and keep signing forever. The blast radius is “while I’m compromised”, not “until I rotate a key I didn’t know had leaked three years ago”.

Why RSA-4096 and not the Ed25519 I’d normally reach for? Because KMS asymmetric signing doesn’t offer Ed25519, and OpenPGP’s packet format is tied to the algorithm that signed it, so the choice of key spec ripples all the way out to the signature on the wire. RSA-4096 is the strong option KMS does offer, so RSA-4096 is what the workflow is built around. A constraint of the box shaped the cryptography, not the other way round, and I’d rather say so than pretend I picked RSA on purpose.

Minting an OpenPGP key from a key you can’t hold

Here’s the part I find genuinely neat. OpenPGP wants a private key to self-sign its own public key when you generate it. And I don’t have a private key in any form I can hand to a library… it’s sitting in KMS, behind a door with no handle on my side. So how do you produce a valid OpenPGP public key at all?

go-tool-base leans on a small Go interface, crypto.Signer: anything that can return its public key and sign a digest. A KMS-backed signer satisfies it by turning each Sign call into a kms:Sign request. Then pkg/openpgpkey builds the OpenPGP entity around that signer:

func Entity(signer crypto.Signer, name, email string, creationTime time.Time) (*openpgp.Entity, error) {
	rsaPub, ok := signer.Public().(*rsa.PublicKey)
	// ...
	pubPkt := packet.NewRSAPublicKey(creationTime, rsaPub)
	// Construct the private-key packet directly (rather than
	// packet.NewSignerPrivateKey, which panics on opaque signers):
	// the crypto.Signer drives the actual signing, so a KMS-backed
	// signer works here.
	privPkt := &packet.PrivateKey{PublicKey: *pubPkt, PrivateKey: signer}
	// ...
}

Look at that PrivateKey packet. The field where OpenPGP expects the secret key material holds the crypto.Signer instead, which is to say, a remote handle to KMS. When the entity self-signs its public key, that self-signature is computed by KMS. gtb keys mint runs exactly this and writes out an ASCII-armored OpenPGP public key, and at no point did a single byte of private key material exist on the machine that minted it. The OpenPGP “private key” is a phone line to a vault, not a key.

That public key is what gets published off-platform over WKD and baked into the binary, the two trust anchors that post cross-checks.

Access without a human and without a standing key

A key that never leaves KMS is only as good as the rules about who may call kms:Sign. The signer role is deliberately narrow: it can call kms:Sign and kms:GetPublicKey on this one key and nothing else, and it is assumable only over OIDC from specific CI subjects, the same keyless federation the rest of the estate runs on. No human holds it. No long-lived access key sits in a CI variable waiting to leak. A release job federates in for its few minutes, signs, and the credentials evaporate with the runner.

So the chain of “who can sign a release” has no standing secret in it anywhere. Not a key file, not an access key, not a console user. Just a short-lived token, scoped to two API calls, on a key that can’t be exported.

The real cost: rotation is manual

This isn’t free, and the bit it taxes you on is rotation. KMS won’t auto-rotate an asymmetric SIGN_VERIFY key, which is why the module sets enable_key_rotation = false rather than leaving a default on. Rotating means minting a new key (a -v2 alias), publishing its public key alongside the old one, and running a dual-sign window long enough that clients have picked up the new anchor before you retire the old. It’s manual, it’s a runbook, and pretending otherwise would be the kind of thing this series exists to argue against. The trade I made was: a key with no exfiltration path, in exchange for rotation I have to do by hand. For a release-signing key, that’s the right side of the trade.

Why this is a command and not a script I hid

The origin of all this is a good deal less tidy than the result. I was working through the key-generation runbook, creating the offline rotation key with a gpg command I’d copied straight off my own page… and it just hung. No error, no prompt, just a cursor blinking while gpg waited on something it never bothered to mention.

My first instinct was the lazy one: drop the minting script into a scripts folder in my infra repo and never speak of it again. Then it nagged. That repo’s private, so the recipe would live somewhere nobody else could ever reach, and I’d already half-promised myself a tutorial walking people through this exact setup. So it shouldn’t sit in infra at all. It should be a gtb command, with a pluggable backend so anyone can swap my KMS for whatever provider they happen to run.

The deeper objection is the one that actually shaped it, though. I didn’t want to be shelling out to gpg by hand in the first place. gtb is a tool I hand to other people, and every time it drops to the shell for some gpg incantation, that’s an environment I’m asking the next person to reproduce, a dependency to install, a fiddly step to get subtly wrong, all before they can sign a single thing. The aim was to keep as much of this inside the box as I could: mint the key, build the WKD tree, produce the signature, all in pure Go, with no gpg on the path and no gpg-wks-client either.

So gtb keys mint pulls the public half out of your KMS key and frames it as OpenPGP, the trick from earlier; gtb keys wkd builds the tree ready to upload; and gtb sign produces the detached signature through that same remote round-trip. What comes out is an entirely ordinary OpenPGP signature gpg --verify is happy with, so you’re not locked into anything of mine. And none of it is just for me: build your tool on go-tool-base and the same handful of commands stands you up with this exact model, pointed at your own KMS. No cloud KMS to hand? There’s a local backend, a plain key on disk, to wire the whole thing together on your laptop first. These are commands for you, the person shipping the tool. Your users never run mytool keys mint… they just get updates that quietly check themselves, which was the whole idea two posts ago.

That setup deserves a walkthrough of its own, and it’ll get one. For now, the ergonomics were the point, not a nicety bolted on afterwards. The safest setup in the world is no use to anyone if it takes a PhD to stand up.

Where this leaves the whole story

Step back and the full loop is finally closed. The private key is born in KMS and never leaves it. Its public key is minted from it, with KMS computing its own self-signature. That public key is published off-platform and embedded in the binary. Releases are signed by KMS, reached only through short-lived OIDC federation. And the client verifies against the embedded and WKD keys cross-checked against each other. At no single point in that chain is there a thing an attacker can grab that lets them forge a release, and the most dangerous thing of all, the private key, has no theft path because it has no export path.

That’s the thread running through the whole signing series, from the very first checksum to here: the strongest control isn’t a better lock on the key. It’s arranging things so the key was never somewhere you could lose it. Nobody is coming to clean your supply chain, so the least you can do is leave it nothing worth stealing.

A signature the platform can't forge

Tue, 09 Jun 2026 00:00:00 +0000

A self-updating tool has a chicken-and-egg problem baked into it. The thing doing the updating is the thing being updated, so when it reaches out and pulls down a newer version of itself, it’s the one that has to decide whether to trust what just landed. No human in the loop, nobody to ask. I’ve been closing that gap in go-tool-base’s self-updater in two phases. The first gave it a checksum: download the new binary, hash it, compare it against the release’s checksums.txt. That catches the accidents, the truncated download, the flipped bit on a dodgy mirror. And I said at the time, plainly, that it does nothing about a determined attacker who owns the release platform… the checksums file sits right next to the binary, so whoever can swap one can swap both. I left that as an IOU. This second phase is me paying it.

The thing a checksum can’t do

A checksum is a promise that the bytes you got match the manifest. It says nothing about who wrote the manifest. So if GitLab, or my account, or a leaked CI token gets compromised, the attacker rewrites the binary and the checksums.txt in the same breath, and the hash matches perfectly, because they’re the one who computed it. It’s the same wall I keep walking into whenever I think about supply-chain trust: a checksum is only ever as good as whatever’s standing behind it, and the thing standing behind a checksum is the very platform that just handed you the file. Same hands, both times.

To get past that, you need a signature whose root of trust lives somewhere the platform can’t reach.

The crypto is the easy part

Here’s the bit that caught me slightly off guard while I was building this: the cryptography is the easy part. Verifying a detached OpenPGP signature is a library call, and go-tool-base’s TrustSet wraps it up in one method:

func (t *TrustSet) VerifyManifestSignature(manifest, signature []byte) error {
	// ...
	signer, err := openpgp.CheckArmoredDetachedSignature(
		t.entities, bytes.NewReader(manifest), bytes.NewReader(signature), nil)
	if err != nil {
		return errors.Wrap(ErrSignatureInvalid, err.Error())
	}
	if signer == nil {
		return errors.Wrap(ErrSignatureInvalid, "no signer in trust set matched")
	}
	return nil
}

Hand it the manifest, the detached signature, and a set of trusted public keys (the entities), and it tells you whether any one of them signed it. That’s the whole of the cryptography, and it’s genuinely not where the hard work lives.

The hard work is that set of trusted public keys. Where do they come from? Because if the answer is “we ship them right next to the binary”, well… you’re straight back to the checksum problem. Whoever can swap the binary can swap the key too, sign with their own, and the check waves it through none the wiser.

Pulling the two questions apart

So the design splits along exactly that seam. The verification half is fixed, and deliberately boring (the method above). The trust anchor, the actual keys, comes from a swappable KeyResolver:

// The interface separates "where the trust anchor comes from" from "how a
// signature is verified against it", so SelfUpdater can be wired with
// whichever resolver chain a tool needs without changing verification logic.
type KeyResolver interface {
	Name() string
	Resolve(ctx context.Context) (*TrustSet, error)
}

That little seam is really the whole game. Everything interesting about standing up to a compromised platform comes down to which resolver you hand the updater, and the verification code never has to know the difference.

Three answers to “where does the key live”

The first option is to embed it. Bake the public key straight into the binary at build time (NewEmbeddedResolver), so it rides along inside a release you already trusted enough to run. Tidy and self-contained. The catch is that a future malicious release could embed a different key, so on its own, embedding really just trusts whoever cut the most recent binary.

The second is WKD, the Web Key Directory. Fetch the key over HTTPS from a well-known path on a domain you control (NewWKDResolver), nothing to do with where the release itself is hosted. Now the key isn’t in the binary at all, so poisoning a release doesn’t touch it. You haven’t made the problem disappear, mind… you’ve moved the trust onto your domain’s host and its DNS. A different blast radius, but a blast radius all the same.

The third option is to do both, and make them agree. Run embedded and WKD, and insist they agree:

func (c *CompositeResolver) Resolve(ctx context.Context) (*TrustSet, error) {
	// ... run each child resolver concurrently ...
	if err := checkAgreement(successes); err != nil {
		return nil, err // ErrKeyResolverMismatch
	}
	return successes[0].ts, nil
}

Think of it as the two-key rule on a safe deposit box, or two witnesses who’ve never met telling you the same story. One source on its own you might quietly doubt. But if the key baked into the binary and the key sitting on my domain hand back the same fingerprint, that agreement is worth a great deal more than either of them alone. And if they ever come back different, that’s not a maybe, that’s an alarm: ErrKeyResolverMismatch. Poison one source and the mismatch is the thing that gives the game away.

That composite is the real answer, and it’s why the interface exists at all. There’s nothing a single attacker can get their hands on that holds the whole thing up by itself. The key is baked into a release you trusted, and fetched from a domain well off the release platform, and the two have to match before a single byte of the update is allowed through.

The separation is the whole point

It’s easy to nod along at “two sources” and miss the part that actually does the work. The agreement between the embedded key and the WKD key is only worth something if an attacker can’t reach both of them from the same place. If the key I bake into the binary and the key I serve over WKD both came out of the same release pipeline, whoever owns that pipeline swaps the pair of them, the fingerprints still match, and the cross-check happily waves the forgery through. Same hands, both times. Again.

So they don’t share a pipeline, and that’s the entire design, not an accident of how things ended up. The binary, and the key embedded in it, are built and signed in GitLab CI, which federates into AWS KMS to do the signing itself. The WKD key lives somewhere else completely: a Cloudflare Pages site serving openpgpkey.phpboyscout.uk, deployed by hand at rotation time with the Wrangler CLI and a token allowed to do nothing but edit that one Pages project. No Git integration, no webhook, nothing that lets a push to the repo or a run of the release pipeline so much as touch it. The Cloudflare account is even administered under a different email and a different second factor from the GitLab and AWS ones, so the three anchors really are independent rather than just feeling that way.

Which is what makes them fail independently, and that independence is the only thing that makes the agreement worth checking. To forge a release that survives the cross-check, an attacker doesn’t have to beat one system, they have to beat two unrelated ones, on different platforms, behind different credentials, in the same window, without either of them noticing.

There’s a quieter benefit in the cadence, too. Releases go out constantly and automatically; the WKD key changes rarely, and only ever by hand. So the busy, automated path, the one an attacker is most likely to prise open, is exactly the one with no power to rewrite the key everyone checks against.

Requiring it, without breaking everyone

Now, a check nobody ever switches on is just theatre. But switch it on before the keys are actually out there in people’s installs, and you’ve handed everyone a self-inflicted outage instead. So the default is deliberately timid. The framework ships DefaultRequireSignature = false: a tool built on go-tool-base doesn’t suddenly start rejecting its own updates the day its author bumps the framework version.

The tool author flips it to true in main(), but only after they’ve shipped a release that embeds the key, so every install out there already holds the trust anchor before the first release that insists on one. Ship the key, then turn the lock: the same leave-yourself-a-way-back discipline as any migration you’d like to still have a job after. And the end user still gets an override (update.require_signature, or an env var) for the day it all goes sideways and they need out.

What it actually buys

The first phase stopped accidents. This one stops the platform. And not because the cryptography is clever, OpenPGP checks the signature in a single call, but because the trust anchor is arranged so that nothing the attacker can actually reach holds the whole thing up on its own. A signature only ever proves the sender, never the contents. All of this is really about making “the sender” something a compromised release host can’t quietly fake its way into being.

Which leaves one last thread dangling. The verifying key gets fetched from somewhere, fine… but the signing key, the private half that actually produces these signatures, has to live somewhere the platform can’t reach either, or none of the rest holds up. That’s the capstone, and where this series ends: where that key lives, and why it never leaves the box it’s born in.

Telemetry that asks, and telemetry that doesn't

Thu, 04 Jun 2026 00:00:00 +0000

go-tool-base has had a thing called telemetry for a long while now. It’s the opt-in kind: the product analytics that asks a user’s permission before it phones a single byte home, sits there as a no-op until they say yes, and can be wiped on request. The whole package is built around consent.

Then the web-service series went and needed telemetry too. Not that telemetry. The other one, the one the rest of the industry means when it says the word: traces, metrics and logs of a running service. And the awkward thing about those two is that they share a name, they want to share a package, and they pull in exactly opposite directions on the one question that matters most.

This is the story of how 0.7.x grew a second telemetry without breaking the first, and where the line between them ended up getting drawn.

Why bother putting it in the framework at all

The starting point is that I could have left observability out. A reader could wire up OpenTelemetry in their own service and go about their day. But the six parts of the web-service series spent a lot of effort making the transports first-class: a gRPC server, an HTTP server, a gateway, TLS across all of them, each one a Register call against the controller. Turning a CLI into a real long-running service and then shrugging “observability is your problem” would have left a hole exactly where it hurts.

Because a service you can’t see into is a liability the moment it leaves your laptop. The series ended with a macguffin service that was typed, fast and served over TLS, and was also a black box: when it got slow, you had nowhere to look. Metrics and traces are how you get the lights on, and they deserved the same first-class treatment as the things they observe.

The other half of the reason is that the framework already had a foot in this world. The analytics package’s preferred backend speaks OTLP, the OpenTelemetry wire protocol. So OpenTelemetry was already in the building. Doing observability any other way would have meant two standards where one would do.

The catch: two telemetries, opposite instincts

Here’s where it gets interesting, and it’s the part worth slowing down on.

The analytics telemetry is about a user. It collects usage data, hashed machine id, which command ran, exit code, and the entire design assumes you have to ask first. It is off by default. The collector you get when it’s disabled is a no-op, so nothing is recorded until the user opts in, and there’s a deletion path for when they change their mind. That’s not an add-on, that’s by design.

The observability telemetry is about a service. It emits operational data, how long a request took, which span was slow, how many errored, to a collector the operator runs. And there is no user in the loop to ask. The operator deploys the service, points it at their collector, and that act is itself the consent. Asking would be nonsensical: whose permission, for data about their own service, on their own infrastructure?

So you have two things called telemetry, wanting to live in one package, with the opposite default on consent. One is off until someone says yes; the other is on the moment it’s configured. Get that wiring wrong and you fail in one of two ugly ways. Gate the operational telemetry behind the user’s analytics opt-in, and a service’s tracing silently does nothing because nobody ticked a box meant for something else. Or loosen the analytics gate to make observability flow, and you start leaking usage data the user never agreed to share. Neither is acceptable, and “just use two packages” throws away everything the two genuinely have in common.

Quite a lot, as it turns out, and all of it below the consent line.

Both ship their data over OTLP to a collector. Both need to describe who is emitting, the service name and version, the resource in OpenTelemetry’s terms. Both parse an endpoint, attach headers, decide whether the connection is plaintext. None of that has the faintest thing to do with consent. It’s just the plumbing of getting bytes to a collector, and the analytics backend already had all of it, written inline.

So the shape of the solution fell out of the problem. Lift the shared plumbing into one place, let both telemetries stand on it, and keep the consent decision firmly out of that shared layer. The structure under pkg/telemetry ended up like this:

pkg/telemetry/
 telemetry.go the analytics Collector (consent-gated)
 backend_otel.go its OTLP backend
 posthog/ datadog/ vendor analytics backends
 otelcore/ shared: OTLP endpoint, resource, telemetry.* config
 tracing/ observability signal
 metrics/ observability signal
 logs/ observability signal
 observability.go Setup: builds the enabled signals (implied consent)

The new otelcore is the keystone. It holds the three things both sides need and nothing they don’t: ParseEndpoint for the OTLP URL, Resource for the service identity, and Resolve for reading the shared telemetry.* config (a base endpoint, plus per-signal overrides, in the same cascade as the TLS config). It imports no signal exporter and knows nothing about traces, metrics, logs or analytics. It is deliberately dumb plumbing.

The refactor: making the old telemetry stand on the new core

This next part is where the old telemetry and the new one become a single thing. The analytics OTLP backend was the first user of OTLP in the framework, and it had grown its own copy of all that plumbing: a function that parsed the endpoint URL, split out the host and path, worked out the insecure flag, and built the resource from a service name. Exactly the code the three new signals were about to need.

So rather than write it a second time and let the two drift, the analytics backend was refactored onto otelcore. Its exporter builder, buildOTelExporterOpts, now calls otelcore.ParseEndpoint, the same function tracing, metrics and logs call, and the resource comes from otelcore.Resource, the same one they use. One implementation of “talk OTLP to a collector”, four callers: the analytics backend and the three observability signals. Change how the framework forms an OTLP endpoint, and every signal moves together.

The reassuring part was that the analytics tests didn’t budge. The refactor moved code without changing behaviour, and the consent machinery, the opt-in, the no-op-when-disabled, the deletion path, never came near otelcore. Which is exactly the point.

Where the line is

Because the shared core is the easy half. The half that earns its keep is the bit that isn’t shared, and it’s a single, deliberate line.

The analytics collector keeps its gate. The constructor, NewCollector, still returns a no-op the moment telemetry is disabled, so a user who hasn’t opted in gets a collector that silently discards everything. Informed consent, untouched.

Observability gets a different door entirely. Setup builds whichever signals the operator has switched on, and it is gated only by telemetry.tracing.enabled and its siblings, which the operator sets. It never consults the analytics opt-in. Turning on tracing doesn’t turn on analytics; disabling analytics doesn’t silence tracing. The two enable flags live under the same telemetry.* config root, sit next to each other, and never read each other.

So that’s the whole architecture in a sentence: one package, one OTLP export core, two consent models that share everything except the answer to “do we need to ask”. The principle underneath, the one that decided every one of these calls, is that the kind of data sets the consent model. Usage data about a person needs informed consent. Operational data about a service runs on implied consent. The CLI and the web service are just where each kind tends to live.

Where this leaves the framework

0.7.x came out the other side with both telemetries: the one that asks first, exactly as it was, and a new one that doesn’t, because it has nobody to ask. They share an export core, a config root and a name, and they part company on the only thing they were ever going to disagree about.

I’ve been careful here to describe how the two consent models are kept apart, not to argue why they have to be. That argument, that “the kind of data decides the consent model” is a line worth holding rather than a convenient bit of engineering, is a piece of its own, and it’s the one I’m writing next.

Building a web service with go-tool-base, part 6: seeing what your service is doing

Tue, 02 Jun 2026 00:00:00 +0000

On paper the macguffin service is finished. Part 5 left it typed, fast, documented and served over TLS. So you deploy it, traffic starts flowing, and a week later someone wanders over to say “it’s slow”.

Slow how? Slow where? Which endpoint, the gateway or the gRPC behind it, the store or the network in between? You open the logs and there’s a wall of “request completed” lines, no way to tell which ones belong to the same request, and not a word about the half-second one of them spent somewhere inside. The service is a black box that happens to be on fire.

That’s the gap this part closes. A long-running service needs to be observable: you need to see what it’s doing, how often, how fast, and where the time actually goes. And we get there without bolting on a separate stack, because go-tool-base speaks OpenTelemetry, and three signals fall out of the same wiring.

This part is written against go-tool-base v0.7.1, the release that brings the observability components in.

The three things you actually want to see

“Observability” is a grand word for three fairly down-to-earth questions. Logs answer what happened (you’ve had those since part 1). Metrics answer how often, and how fast: request rates, error counts, the spread of latencies. And traces answer where did the time go, by following a single request as it crosses from one piece of your system to the next.

For a service shaped like ours, that last one is the prize. A REST call hits the gateway, turns into a gRPC call, runs through the store, and comes back. A trace stitches that whole journey into one timeline, so “it’s slow” stops being a shrug and becomes “it’s slow in CreateMacguffin, in the store, for 400ms”. You can’t get there from a pile of disconnected log lines.

OpenTelemetry (OTel, if you’ve seen it bandied about) is the vendor-neutral standard for all three. You instrument once and ship the data to whatever backend you fancy (Jaeger, Grafana, Honeycomb, a plain collector) over one wire protocol, OTLP. go-tool-base does the instrumenting; you point it at a collector and pick your poison.

Turn it on

One call wires the lot. In the serve command from part 4, right after the controller exists:

// pkg/cmd/serve/main.go

import "gitlab.com/phpboyscout/go-tool-base/pkg/telemetry"

controller := controls.NewController(ctx, controls.WithLogger(p.Logger))

// Build whichever signals are enabled, install them, flush them on shutdown.
if _, err := telemetry.Setup(ctx, p, controller); err != nil {
	return err
}

telemetry.Setup reads your config, builds whichever of the three signals you’ve switched on, and installs them as the process-wide OTel providers. It also hangs their shutdown on the controller, so when a SIGTERM lands the buffered spans and metrics flush before the process exits. That’s the same graceful-shutdown promise from part 1, now extended to your telemetry: a clean stop drops nothing, not even the trace that was mid-flight.

On its own that line shows you nothing. A signal you haven’t enabled is skipped, so an unconfigured service pays exactly nothing. You turn each on in config, which we’ll get to. First, the instrumentation.

Wire it onto the transports

Setup builds the pipeline; two short additions feed it. The gRPC server takes a stats handler:

// pkg/cmd/serve/main.go

grpcSrv, err := gtbgrpc.Register(ctx, "grpc", controller, p.Config, p.Logger,
	gtbgrpc.OTelStatsHandler())

and the HTTP server takes a middleware. This is also where we finally meet go-tool-base’s middleware chain, which we’ve managed without until now. A chain is just an ordered stack of wrappers around your handler. We put the OTel one first so it opens the span, and the request logger second, so it runs inside that span and can see it:

// pkg/cmd/serve/main.go

chain := gtbhttp.NewChain(
	gtbhttp.OTelMiddleware("macguffin"),
	gtbhttp.LoggingMiddleware(p.Logger),
)

if _, err := gtbhttp.Register(ctx, "http", controller, p.Config, p.Logger, mux,
	gtbhttp.WithMiddleware(chain)); err != nil {
	return err
}

That’s the whole code change. Both helpers, OTelMiddleware and OTelStatsHandler, lean on the standard OTel instrumentation libraries underneath (otelhttp and otelgrpc), so the spans and metrics they emit follow the conventions every backend already knows how to read. Every request now opens a span and records its own duration, on both transports, and you wrote none of it per handler.

Go deeper than the plumbing

The transport spans show the request crossing HTTP and gRPC, but they stop at the edge of your handler. To see where the time goes inside it, you open a span yourself, and that needs no go-tool-base API at all, just OpenTelemetry. Here’s the store doing it, opening a span around the work and tagging it with what it’s handling:

// internal/macguffin/store.go

import (
	"go.opentelemetry.io/otel"
	"go.opentelemetry.io/otel/attribute"
)

var tracer = otel.Tracer("macguffinsvc/store")

func (s *Store) Create(ctx context.Context, name string, quantity int32) Macguffin {
	_, span := tracer.Start(ctx, "Store.Create")
	defer span.End()

	span.SetAttributes(
		attribute.String("macguffin.name", name),
		attribute.Int("macguffin.quantity", int(quantity)),
	)

	// ... store the macguffin, generate its id ...

	span.SetAttributes(attribute.String("macguffin.id", id))

	return m
}

The span starts as a child of whatever is already in the context, which is the gRPC server span the handler is running under, so it slots into the trace on its own. The one thing to get right is the context: give the store method a ctx context.Context first parameter and thread it down from the handler, and the span finds its parent. Tag it with the things you’d want when something’s wrong, the macguffin’s name and its id, and they show up on the span in the UI, right next to the timing.

Point it at a collector

The wiring’s on; now tell it where to send. The config lives under one root:

# config.yaml
telemetry:
 endpoint: "http://localhost:4318"
 insecure: true # plaintext: a local collector only
 tracing:
 enabled: true
 sampling: 1.0 # every trace while you're looking
 metrics:
 enabled: true
 logs:
 enabled: true

endpoint is your OTLP collector. Sampling defaults to a level-headed 0.1 in production (you rarely want every trace under real load), so turn it up to 1.0 while you’re developing or you’ll wonder where yours went. Stand up a collector (a local Jaeger handles traces and gives you a UI to poke at), start the service, and push some traffic through the REST side:

$ curl https://localhost:8443/v1/macguffins
$ curl -X POST https://localhost:8443/v1/macguffins -d '{"name":"the-grail","quantity":1}'

One request, the whole way through

Open the collector and there it is, the thing that justifies the exercise: a single trace for that POST, four spans deep. The HTTP server span on top, the gRPC call nested inside it, the gRPC handler inside that, and your Store.Create span at the bottom, with the macguffin’s name and id sat right on it. The REST request became a gRPC call, crossed the gateway we built in part 4, ran the handler and went into the store, and OTel followed it the entire way, because the trace context travels with the request and everything reads from the same providers. “It’s slow” finally has somewhere to point, and “slow where” has an answer.

The metrics ride the same pipeline: request counts, error counts, a latency histogram broken down by route and method. And your logs get an upgrade you didn’t ask for. Every “request completed” line now carries the trace_id and span_id of the request it describes, so a slow trace in the UI and its log line in your terminal share an id, and you can jump straight from one to the other. That correlation lands in your stderr logs and in the records shipped to the collector, both.

A word on the other telemetry

If you read Telemetry that asks first, you’ll know go-tool-base has a telemetry package for product analytics too: the opt-in kind that asks a user’s permission before it phones anything home. This is not that, and the difference is the whole point.

That analytics is about a user, and it’s gated behind informed consent: off until they say yes. This is about your service, emitted by you, to a collector you run, and it works on implied consent. You switched it on; there’s no end user in the loop to ask. Same package, deliberately separate gates, and never the twain shall meet. There’s a fuller post in that one distinction, and it’s the next thing I’m writing.

Where that leaves the service

Six parts in, the macguffin service starts cleanly, reports its health, serves gRPC and REST and interactive docs over TLS, shuts down without dropping a request, and now tells you what it’s doing while it runs, all from one annotated proto and a controller it registers against. Observability was the last missing sense. The thing was fast and correct before, but you couldn’t see inside it. Now you can, and it cost you about a dozen lines.

Building a web service with go-tool-base, part 5: docs that write themselves

Sat, 30 May 2026 00:00:00 +0000

The google.api.http annotations we added in part 4 have done one job so far: they told the gateway which REST calls map to which RPCs. But they describe the API precisely, the paths, the verbs, the request and response shapes, and a precise description of an API is most of an OpenAPI document. Feed that document to a viewer and you have an interactive docs site: every endpoint listed, every field typed, and a “try it” button that calls the real service.

So from one .proto we’ve had gRPC, then REST, and now documentation, none of it a separate thing to write or keep in sync.

The annotations, a third time

The pattern of this whole series is one source of truth and several outputs generated from it. The proto defined the gRPC service; the annotations on it generated the REST gateway; and those same annotations generate an OpenAPI document. Add a field to a message, and it shows up in the gRPC API, the REST API, and the docs, all at once, because all three are read from the proto.

Generate the OpenAPI document

This is one more buf plugin. A small wrinkle to know up front: grpc-gateway ships its own OpenAPI generator, but it emits OpenAPI v2 (the old Swagger format). For a v3 document we use kollalabs/protoc-gen-openapi, which reads the very same google.api.http annotations:

go install github.com/kollalabs/protoc-gen-openapi@latest

# buf.gen.yaml
plugins:
 # ... the go, go-grpc and grpc-gateway plugins from before ...
 - local: protoc-gen-openapi
 out: internal/docs/assets
 opt:
 - title=Macguffin API
 - default_response=false

go generate ./... now also writes an openapi.yaml. It’s the REST API described in full, drawn straight from the annotations:

# internal/docs/assets/openapi.yaml (generated, trimmed)
openapi: 3.0.3
info:
 title: Macguffin API
 version: 0.0.1
paths:
 /v1/macguffins:
 get:
 summary: ListMacguffins
 operationId: MacguffinService_ListMacguffins
 parameters:
 - name: pageSize
 in: query
 schema:
 type: integer
 format: int32
 responses:
 "200":
 description: OK
 content:
 application/json:
 schema:
 $ref: '#/components/schemas/ListMacguffinsResponse'

That GET /v1/macguffins operation is the get: "/v1/macguffins" rule from the proto, turned into OpenAPI. You wrote the annotation once; it now feeds three generators.

Serve it, viewer and all

A spec on disk is a means, not an end; people want to read it. The usual move is to bolt on a docs UI like Stoplight Elements or Swagger UI, which means vendoring a couple of megabytes of JavaScript into every project. go-tool-base’s openapi package does that part for you: the Stoplight Elements UI is embedded in the framework, so your project ships only its generated spec.

openapi.Register mounts both the document and the docs site onto a mux:

// internal/docs/docs.go

package docs

import (
	_ "embed"
	"net/http"

	"gitlab.com/phpboyscout/go-tool-base/pkg/openapi"
)

//go:embed assets/openapi.yaml
var spec []byte

// Register mounts /openapi.yaml and the Stoplight docs site (/docs/) onto mux.
func Register(mux *http.ServeMux) error {
	return openapi.Register(mux, spec, openapi.WithTitle("Macguffin API"))
}

The //go:embed bakes the generated spec into the binary (the same trick as part 3.5), and openapi.Register serves it at /openapi.yaml and the Stoplight site at /docs/. Wiring it into serve is one line, on the same mux the gateway is mounted on:

// pkg/cmd/serve/main.go (the existing HTTP wiring)

mux := stdhttp.NewServeMux()
mux.Handle("/v1/", gw) // REST, from part 4

if err := docs.Register(mux); err != nil {
	return err
}

Why it goes on the same server

That last point is doing more work than it looks. The docs, the spec, and the live REST API are all on the one HTTP server, so they share an origin. That’s what makes the “try it” console actually work: when you fill in a request in the docs and hit send, the browser calls /v1/macguffins on the same host it loaded the page from, with no cross-origin dance to configure. And because the certificate is your mkcert one, the page and its requests are all clean HTTPS, no warnings to click past, which is exactly why we set the local CA up back in part 2.

$ curl https://localhost:8443/openapi.yaml | head -4
# Generated with protoc-gen-openapi
openapi: 3.0.3
info:
 title: Macguffin API

Open https://localhost:8443/docs/ in a browser and there’s the service: every endpoint, every field, and a working console that calls the real thing.

One proto, three faces

Step back and look at what a single annotated .proto is now producing: a gRPC service for the things that speak it, a REST API for the things that don’t, and an interactive docs site for the people who have to consume either. One source, three faces, and nothing hand-maintained between them. That’s the whole argument for building it this way, and it’s why the annotations were worth the small ceremony.

The service is, by any reasonable measure, done: typed, fast, documented, and served over TLS. The last part is about what happens once it’s out there and taking traffic. In part 6 we add telemetry and logging, so you can see how it’s being used and why it’s slow, without bolting on a separate observability stack.

Building a web service with go-tool-base, part 4: REST for free, with the gateway

Fri, 29 May 2026 00:00:00 +0000

A quick tally of where part 3 left us. One domain, the Store. One gRPC service over it, mapping the domain to proto with toProto. And then a whole second transport, the REST layer, with its own routing and its own toDTO mapping the very same domain into the very same shape, by hand. Two encodings of one thing, drifting apart the moment anyone adds a field and forgets the other side.

I promised that doubling would go away. This is the part where it does, and the thing that does it is the grpc-gateway.

What the gateway actually is

The grpc-gateway is a reverse proxy, generated from your .proto, that speaks REST on the front and gRPC on the back. A JSON request comes in, the gateway turns it into the matching gRPC call, hands it to your gRPC server, and turns the gRPC response back into JSON on the way out.

Read that again with part 3 in mind, because it’s the whole point. The gateway does the JSON-to-proto-and-back encoding for you, using the proto types your gRPC server already produces. You wrote domain → proto once, in the gRPC adapter. The gateway supplies proto → JSON. There is no second hand-written encoding to keep in step, because there is no second implementation: REST becomes a generated front door onto the gRPC service you already have.

So the plan is short. Tell the proto which HTTP calls map to which RPCs, regenerate, wire the gateway in, and delete the part-3 REST layer entirely.

Map HTTP onto the proto

gRPC has no opinion about URLs and verbs; REST is all URLs and verbs. The bridge is an annotation, google.api.http, that you attach to each RPC to say “this one is GET /v1/macguffins/{id}”. Here’s the service with those rules added:

// proto/macguffin/v1/macguffin.proto

import "google/api/annotations.proto";

service MacguffinService {
 rpc GetMacguffin(GetMacguffinRequest) returns (Macguffin) {
 option (google.api.http) = {get: "/v1/macguffins/{id}"};
 }
 rpc ListMacguffins(ListMacguffinsRequest) returns (ListMacguffinsResponse) {
 option (google.api.http) = {get: "/v1/macguffins"};
 }
 rpc CreateMacguffin(CreateMacguffinRequest) returns (Macguffin) {
 option (google.api.http) = {
 post: "/v1/macguffins"
 body: "*"
 };
 }
}

Each rule is small but exact. {id} in the path binds to the id field of the request message. body: "*" on the create says the whole JSON body maps onto the request. The list takes no body and no path parameter, just the verb and path. This is the same information part 3’s hand-written routes carried, except now it lives next to the RPC it describes, and a generator reads it instead of you.

These rules go a good deal further than the three cases we need: query-string parameters, several URL bindings for a single RPC, custom verbs, choosing which field becomes the response body. When you reach for those, the grpc-gateway docs walk through the mapping, and the canonical reference is the HttpRule message that google.api.http comes from, its comments document every option.

The annotations.proto import comes from Google’s common protos, so tell buf where to find them by adding a dependency, then fetch it:

# buf.yaml
deps:
 - buf.build/googleapis/googleapis

buf dep update

Generate the gateway

This is another buf plugin, exactly like part 2’s. Install it:

go install github.com/grpc-ecosystem/grpc-gateway/v2/protoc-gen-grpc-gateway@latest

and add it to the generators:

# buf.gen.yaml
version: v2
plugins:
 - local: protoc-gen-go
 out: internal/gen
 opt: paths=source_relative
 - local: protoc-gen-go-grpc
 out: internal/gen
 opt: paths=source_relative
 - local: protoc-gen-grpc-gateway
 out: internal/gen
 opt: paths=source_relative

go generate ./... now also writes macguffin.pb.gw.go, the gateway: a RegisterMacguffinServiceHandler function that, given a connection to your gRPC server, mounts the REST routes the annotations described.

Wire it on

The gateway needs to call your gRPC server, which means dialling it like any other client, over the same TLS, with credentials that trust its certificate. That’s fiddly to get right by hand, so go-tool-base’s gateway package does it for you. gateway.New opens the connection (matching your server’s transport security) and hands you a mux to register the generated handlers on:

// pkg/cmd/serve/main.go

package serve

import (
	"context"
	stdhttp "net/http"

	"github.com/grpc-ecosystem/grpc-gateway/v2/runtime"
	"google.golang.org/grpc"

	"gitlab.com/phpboyscout/go-tool-base/pkg/controls"
	"gitlab.com/phpboyscout/go-tool-base/pkg/gateway"
	gtbgrpc "gitlab.com/phpboyscout/go-tool-base/pkg/grpc"
	gtbhttp "gitlab.com/phpboyscout/go-tool-base/pkg/http"
	"gitlab.com/phpboyscout/go-tool-base/pkg/props"

	macguffinv1 "gitlab.com/myorg/macguffinsvc/internal/gen/macguffin/v1"
	"gitlab.com/myorg/macguffinsvc/internal/grpcsvc"
	"gitlab.com/myorg/macguffinsvc/internal/macguffin"
)

func RunServe(ctx context.Context, p *props.Props, _ *ServeOptions, _ []string) error {
	controller := controls.NewController(ctx, controls.WithLogger(p.Logger))

	store := macguffin.NewStore()

	// gRPC: the one implementation, mapping the domain to proto.
	grpcSrv, err := gtbgrpc.Register(ctx, "grpc", controller, p.Config, p.Logger)
	if err != nil {
		return err
	}

	macguffinv1.RegisterMacguffinServiceServer(grpcSrv, grpcsvc.New(store))

	// REST, for free: the gateway proxies JSON/HTTP to the gRPC server above.
	gw, err := gateway.New(ctx, p.Config,
		func(ctx context.Context, mux *runtime.ServeMux, conn *grpc.ClientConn) error {
			return macguffinv1.RegisterMacguffinServiceHandler(ctx, mux, conn)
		})
	if err != nil {
		return err
	}

	mux := stdhttp.NewServeMux()
	mux.Handle("/v1/", gw)

	if _, err := gtbhttp.Register(ctx, "http", controller, p.Config, p.Logger, mux); err != nil {
		return err
	}

	controller.Start()
	controller.Wait()

	return nil
}

The only macguffin-specific line is the one inside the callback, RegisterMacguffinServiceHandler. Everything around it, the dial, the credentials, the mux, is the framework’s. Mount the result under /v1/, register it on the same controller and HTTP server as before, and you’re done.

Delete the duplication

Here’s the satisfying bit. The hand-written REST adapter from part 3, the resthand package, the routes, the toDTO, all of it, comes out. You don’t need it: the gateway serves the same REST surface, backed by the gRPC service, from the proto. The serve command shrinks to one gRPC server and one gateway, and your codebase now has a single place where a macguffin becomes JSON.

See it work

The gateway answers REST, and it’s the same store the gRPC service uses:

$ curl https://localhost:8443/v1/macguffins
{"macguffins":[{"id":"m-1","name":"maltese-falcon","quantity":1}]}

$ curl -X POST https://localhost:8443/v1/macguffins -d '{"name":"the-grail","quantity":1}'
{"id":"m-2","name":"the-grail","quantity":1}

Create over REST, and the macguffin is there over gRPC a moment later, because both are the same implementation over the same Store:

$ grpcurl localhost:50051 macguffin.v1.MacguffinService/ListMacguffins
{ "macguffins": [ { "id": "m-1", ... }, { "id": "m-2", "name": "the-grail", ... } ] }

Errors, and changing them

Error handling comes across too. When a gRPC handler returns a status code, the gateway maps it to the matching HTTP status. The codes.NotFound we returned back in part 2 arrives as a 404, with a JSON error body, and we wrote none of it:

$ curl -s -o /dev/null -w '%{http_code}\n' https://localhost:8443/v1/macguffins/nope
404

That default mapping is the sensible one you’d reach for anyway. A few of the common codes:

gRPC code	HTTP
`InvalidArgument`	400
`Unauthenticated`	401
`PermissionDenied`	403
`NotFound`	404
`AlreadyExists`	409
`Internal`	500
`Unavailable`	503

So the rule of thumb is simply to return the right codes.* from your gRPC handlers, and the REST side gets the right status for free.

When the default shape isn’t what your clients expect, a {"error": {…}} envelope, a trace id header, a tweak to one particular status, you supply your own error handler. The grpc-gateway takes one as a runtime.ServeMuxOption, and gateway.New passes those straight through:

gw, err := gateway.New(ctx, p.Config, register,
	gateway.WithMuxOptions(runtime.WithErrorHandler(myErrorHandler)),
)

myErrorHandler receives the error and the http.ResponseWriter and writes whatever your API promises. That same WithMuxOptions hatch takes the gateway’s other knobs, header matchers, custom marshalers, and the rest; they’re all in the grpc-gateway docs.

Where this leaves us

This is the shape the series was building towards. One domain, one gRPC implementation, one domain → proto mapping you wrote by hand, and a REST API generated from the same proto that needed no second implementation and no second encoding. The things that speak gRPC get gRPC; the browser, the webhook and the curl get JSON; and there’s exactly one place to change when a macguffin grows a field.

Those google.api.http annotations have one more trick in them. They describe your REST API precisely enough to generate an OpenAPI document, and in part 5 we serve that as a live, clickable docs site, from the very same proto.

Building a web service with go-tool-base, part 3.5: the same server, now a website

Thu, 28 May 2026 00:00:00 +0000

The HTTP server from part 3 serves JSON. But net/http doesn’t care what you hand it: HTML, an image, a stylesheet, a whole little site, it’s all just bytes with a content type. So before we get back to the API in part 4, a short detour to prove the point and pick up a couple of genuinely useful tools: we’ll turn the macguffin service into a tiny website.

This is a bonus, off to the side of the API arc, but it earns its place. Real services nearly always grow a bit of HTML eventually: a status page, a landing page, a small admin view, an embedded docs site (we’ll do exactly that in part 5). The mechanics are the same every time, and worth having in hand.

A page from `html/template`

Go’s html/template renders HTML from a template and your data, and it escapes that data on the way out, so a macguffin called <script> becomes text rather than a problem. Here’s a page that lists the catalogue:

<!-- internal/site/templates/index.html -->
<!doctype html>
<html lang="en">
 <head>
 <meta charset="utf-8" />
 <title>Macguffins</title>
 <link rel="stylesheet" href="/static/style.css" />
 </head>
 <body>
 <h1>The macguffin catalogue</h1>
 <ul>
 {{range .}}
 <li>{{.Name}} <span class="qty">&times;{{.Quantity}}</span></li>
 {{end}}
 </ul>
 </body>
</html>

{{range .}} walks the slice we pass in, and {{.Name}} / {{.Quantity}} read each macguffin’s fields. The data is the same Store from part 2, so the page is a view onto the very same domain the gRPC and JSON APIs serve.

Shipping the files inside the binary

A template and a stylesheet are files, and you do not want to deploy a folder of loose assets next to your binary and hope they line up. Go’s embed package bakes them into the binary at build time, so the whole thing ships as one file.

// internal/site/site.go

package site

import (
	"embed"
	"html/template"
	"io/fs"
	"net/http"

	"gitlab.com/myorg/macguffinsvc/internal/macguffin"
)

//go:embed templates static
var content embed.FS

var tmpl = template.Must(template.ParseFS(content, "templates/index.html"))

type Site struct {
	store *macguffin.Store
}

func New(store *macguffin.Store) *Site {
	return &Site{store: store}
}

func (s *Site) Routes() *http.ServeMux {
	static, err := fs.Sub(content, "static")
	if err != nil {
		panic(err) // the embedded path is a compile-time constant
	}

	mux := http.NewServeMux()
	mux.HandleFunc("GET /{$}", s.index)
	mux.Handle("GET /static/", http.StripPrefix("/static/", http.FileServer(http.FS(static))))

	return mux
}

func (s *Site) index(w http.ResponseWriter, _ *http.Request) {
	if err := tmpl.Execute(w, s.store.List()); err != nil {
		http.Error(w, err.Error(), http.StatusInternalServerError)
	}
}

Three things are doing the work. //go:embed templates static pulls both folders into the content filesystem. template.ParseFS parses the page from it once at startup. And http.FileServer(http.FS(static)) serves the stylesheet (and anything else under static/) straight from the embedded files, with content types set for you, so GET /static/style.css comes back as text/css.

The GET /{$} pattern is worth a note: the {$} anchors it to the exact root path, so / renders the page but /anything-else doesn’t accidentally match it.

If you’d rather edit templates without rebuilding during development, swap the embedded filesystem for the real one: http.FileServer(http.Dir("static")), and template.ParseGlob instead of ParseFS. Embed for release, disk for the edit-refresh loop; the handlers don’t change.

On the same server

Routes() hands back a *http.ServeMux, which is an http.Handler, so it registers exactly like the JSON API did, on the same controller, with the same TLS:

// pkg/cmd/serve/main.go (or a dedicated command)

if _, err := gtbhttp.Register(ctx, "site", controller, p.Config, p.Logger,
	site.New(store).Routes()); err != nil {
	return err
}

Because the certificate is the mkcert one from part 2, opening https://localhost:8443/ renders the page, stylesheet and all, with a clean padlock on any machine that trusts your local CA (where you ran mkcert -install); anywhere else, the browser flags the cert, exactly as it should.

The same hardened server, the same graceful shutdown, the same /healthz, now serving a website instead of (or alongside) JSON.

Back to the API

That’s the whole trick: the HTTP server is just net/http, and it will serve whatever you point it at, escaped and content-typed properly, shipped inside the binary. We’ll use exactly this in part 5 to serve interactive API docs.

Detour over. In part 4 we get back to the API and finally deal with that duplicated REST layer, the one we wrote twice and promised to delete.

Building a web service with go-tool-base, part 3: a REST service, by hand and by spec

Wed, 27 May 2026 00:00:00 +0000

The gRPC service from part 2 is the right core for service-to-service traffic. It is also useless to a great many of the things that might want to call it: a browser, a webhook from some SaaS, a partner who will not touch protobuf, a curl in a 2am runbook. They all want the same thing, plain JSON over HTTP.

So we need a REST face as well. The only way we know how so far is to build one, a second implementation of the macguffin service, this time over HTTP. We’ll do it two ways, by hand and from a spec, and wire it onto the very same controller from part 1. And yes, we’ll feel the duplication, because the next part is about making it disappear.

Why not just `net/http`?

A fair question before we register anything: go-tool-base’s HTTP server is net/http underneath, so why not stand up a http.Server{} yourself? You can, and nothing here hides the standard library from you. What gtbhttp.Register adds is the dull, easy-to-botch scaffolding that goes around the handler:

It’s still net/http. You write ordinary http.Handlers. There’s no bespoke router and no framework to learn underneath, just the standard library doing the serving.
Timeouts and limits you’d otherwise forget. A bare http.Server{} has no ReadTimeout or WriteTimeout, which is a Slowloris waiting to happen. You get sane read/write/idle timeouts and caps on header and body sizes.
The hardened TLS from part 2, the same shared certificate and the same 1.2-minimum AEAD config, with no extra setup.
Lifecycle for free. It registers on the same controller as your gRPC server, so a SIGTERM drains in-flight requests and stops both together, instead of you re-writing the signal-and-Shutdown dance for every service.
Health endpoints, /healthz, /livez and /readyz, backed by the controller and ready for an orchestrator to probe, with nothing to write.

You write the handlers; it owns the lifecycle and the hardening. With that settled, let’s write some handlers.

By hand, on the standard library

net/http’s ServeMux acquired method and path patterns in Go 1.22, so a small REST surface needs no router and no dependencies at all. And the hard part is already behind us: the Store from part 2 is our domain, and the HTTP handlers are just another thin adapter over it, exactly as the gRPC server was. A small type holds the store, and the routes hang off it:

// internal/resthand/server.go

package resthand

import (
	"encoding/json"
	"net/http"

	"gitlab.com/myorg/macguffinsvc/internal/macguffin"
)

type API struct {
	store *macguffin.Store
}

func (a *API) Routes() *http.ServeMux {
	mux := http.NewServeMux()
	mux.HandleFunc("GET /macguffins", a.list)
	mux.HandleFunc("GET /macguffins/{id}", a.get)
	mux.HandleFunc("POST /macguffins", a.create)

	return mux
}

Each pattern names a method and a path, and {id} is a named wildcard a handler reads back with r.PathValue("id"), no third-party router required. And because the domain Macguffin already carries JSON tags, this adapter can encode it straight to the response, with no separate type to map to:

// internal/resthand/server.go (same file)

func (a *API) get(w http.ResponseWriter, r *http.Request) {
	m, ok := a.store.Get(r.PathValue("id"))
	if !ok {
		http.Error(w, "macguffin not found", http.StatusNotFound)
		return
	}

	writeJSON(w, http.StatusOK, m)
}

list and create are the same shape: ask the store, encode the result with a small writeJSON helper. Routes() hands back a *http.ServeMux, which is itself an http.Handler, so it slots straight into the controller in a moment.

There’s nothing clever in any of it, and that’s the appeal: total control, no tooling, and because we serve the domain type directly, nothing to map. The cost is the kind that creeps up on you. Every route, every bit of marshalling, every status code is yours to write and keep correct, and as the surface grows, so does the area for small mistakes.

By spec, with oapi-codegen

The other road is the OpenAPI mirror of part 2’s proto. You describe the API in an OpenAPI document, and a generator turns it into a Go interface for you to implement. The tool is oapi-codegen.

First the contract, api/macguffin.openapi.yaml (trimmed to one path here):

# api/macguffin.openapi.yaml
paths:
 /macguffins/{id}:
 get:
 operationId: getMacguffin
 parameters:
 - name: id
 in: path
 required: true
 schema: { type: string }
 responses:
 "200":
 description: OK
 content:
 application/json:
 schema:
 $ref: "#/components/schemas/Macguffin"
 "404":
 description: Not found
components:
 schemas:
 Macguffin:
 type: object
 required: [id, name, quantity]
 properties:
 id: { type: string }
 name: { type: string }
 quantity: { type: integer, format: int32 }

Install the tool and tell it what to emit with a small config. We want the standard-library server, so std-http-server:

go install github.com/oapi-codegen/oapi-codegen/v2/cmd/oapi-codegen@latest

# api/oapi-codegen.yaml
package: restapi
output: internal/restapi/macguffin.gen.go
generate:
 std-http-server: true
 models: true

As in part 2, wire it into go generate so it can’t drift, this time sitting right alongside the buf directive:

// gen.go (at your module root): add the second line alongside part 2's

//go:generate buf generate
//go:generate oapi-codegen -config api/oapi-codegen.yaml api/macguffin.openapi.yaml

go generate ./... now regenerates both the gRPC and the REST code in one go. What oapi-codegen writes is the message models and, the important part, a ServerInterface, one method per operation:

// internal/restapi/macguffin.gen.go (generated)

type ServerInterface interface {
	ListMacguffins(w http.ResponseWriter, r *http.Request)
	CreateMacguffin(w http.ResponseWriter, r *http.Request)
	GetMacguffin(w http.ResponseWriter, r *http.Request, id string)
}

If that feels familiar, it should: it’s the exact shape from part 2. The generator hands you an interface and your job is a type that honours it, an adapter over the same Store. But here’s the wrinkle that earns this part its keep. oapi-codegen generated its own Macguffin type, a DTO, so the adapter has to map the domain into it. (The path parameter, at least, arrives already typed as id string rather than fished out by hand.)

// internal/restapi/server.go

package restapi

import (
	"net/http"

	"gitlab.com/myorg/macguffinsvc/internal/macguffin"
)

func (a *API) GetMacguffin(w http.ResponseWriter, _ *http.Request, id string) {
	m, ok := a.store.Get(id)
	if !ok {
		http.Error(w, "macguffin not found", http.StatusNotFound)
		return
	}

	writeJSON(w, http.StatusOK, toDTO(m))
}

// toDTO maps the domain type to oapi-codegen's generated Macguffin DTO.
func toDTO(m macguffin.Macguffin) Macguffin {
	return Macguffin{Id: m.ID, Name: m.Name, Quantity: m.Quantity}
}

That toDTO is the second mapping of this kind we’ve written. Part 2’s gRPC adapter had toProto; this one has toDTO. The same domain data, encoded twice, into two generated shapes, kept in step by hand. Hold that thought: part 4 is where that second mapping stops being something you write.

A generated helper turns your implementation into an http.Handler:

handler := restapi.HandlerFromMux(restapi.NewAPI(store), http.NewServeMux())

So which road? By hand when it’s a handful of endpoints you fully control and you’d rather not own a generator. Spec-first when the contract carries weight, several teams consume the API, the shape changes often, or you simply want the OpenAPI document to exist, which (spoiler) we’ll be serving as live, clickable docs in part 5.

Wire it onto the controller

Whichever road you took, you’re holding an http.Handler. We don’t replace the gRPC server from part 2; we add the HTTP one beside it, on the same controller, both reading from one shared store. This is the controller from part 1 doing exactly what it exists for: two transports, one lifecycle.

// pkg/cmd/serve/main.go

package serve

import (
	"context"

	"gitlab.com/phpboyscout/go-tool-base/pkg/controls"
	gtbgrpc "gitlab.com/phpboyscout/go-tool-base/pkg/grpc"
	gtbhttp "gitlab.com/phpboyscout/go-tool-base/pkg/http"
	"gitlab.com/phpboyscout/go-tool-base/pkg/props"

	macguffinv1 "gitlab.com/myorg/macguffinsvc/internal/gen/macguffin/v1"
	"gitlab.com/myorg/macguffinsvc/internal/grpcsvc"
	"gitlab.com/myorg/macguffinsvc/internal/macguffin"
	"gitlab.com/myorg/macguffinsvc/internal/resthand"
)

func RunServe(ctx context.Context, p *props.Props, _ *ServeOptions, _ []string) error {
	controller := controls.NewController(ctx, controls.WithLogger(p.Logger))

	// One shared domain behind both transports.
	store := macguffin.NewStore()

	// gRPC, from part 2.
	grpcSrv, err := gtbgrpc.Register(ctx, "grpc", controller, p.Config, p.Logger)
	if err != nil {
		return err
	}

	macguffinv1.RegisterMacguffinServiceServer(grpcSrv, grpcsvc.New(store))

	// HTTP, new this part. (Or the oapi-codegen handler; either is an http.Handler.)
	handler := resthand.New(store).Routes()
	if _, err := gtbhttp.Register(ctx, "http", controller, p.Config, p.Logger, handler); err != nil {
		return err
	}

	controller.Start()
	controller.Wait()

	return nil
}

The HTTP Register is the counterpart of part 2’s gRPC one: same controller, so a single SIGTERM drains and stops both together and /healthz reports on both. The HTTP server reads server.http.port, the gRPC server server.grpc.port from part 2, and both take their TLS from the shared server.tls block, the one certificate:

server:
 grpc:
 port: 50051
 reflection: true
 http:
 port: 8443
 tls:
 enabled: true
 cert: ./localhost+2.pem
 key: ./localhost+2-key.pem

A browser away

Here’s where the mkcert groundwork from part 2 pays off. Because that certificate is signed by a CA your machine already trusts, the service answers over HTTPS with no --cacert flag and, more to the point, no browser warning:

$ curl https://localhost:8443/macguffins
[{"id":"m-1","name":"maltese-falcon","quantity":1}]

$ curl https://localhost:8443/macguffins/m-1
{"id":"m-1","name":"maltese-falcon","quantity":1}

And /healthz now reports both transports, since they registered against the one controller:

$ curl https://localhost:8443/healthz
{"overall_healthy":true,"services":[{"name":"grpc","status":"OK"},{"name":"http","status":"OK"}]}

Open https://localhost:8443/macguffins in an actual browser and it just loads, green padlock and all. That matters more than it sounds, and it’s the reason we set the local CA up early: in part 5 the API docs are a web page, and a docs page behind a cert warning is a docs page nobody trusts.

Where this leaves us

Step back and count what we actually built. There’s one domain, the Store, and it didn’t change at all this part. What we added was a second delivery adapter over it: routing, marshalling, and, on the spec-first road, a toDTO mapping sitting beside part 2’s toProto. The same data, encoded into two generated shapes, kept in step by hand. That’s the real duplication, not the logic, but the transport scaffolding wrapped around it.

And it’s the itch the next part scratches. We built a second transport and a second encoding of the same domain, and kept them in step by hand. Part 4 is where that whole second layer, adapter and encoding both, stops being something you write at all. We felt the cost first on purpose; now we get to remove it.

Building a web service with go-tool-base, part 2: a gRPC service, with TLS

Tue, 26 May 2026 00:00:00 +0000

The heartbeat from part 1 runs, ticks along, and shuts down politely when you ask it to. It also talks to absolutely no one. A service people can actually call needs an API, and for a typed, fast, streaming-capable one, gRPC is the obvious first move.

The catch is that a production-grade gRPC server is rather more than grpc.NewServer(). You want health checks an orchestrator understands, reflection so you can poke at it without the .proto file in hand, a graceful shutdown that doesn’t guillotine calls that are still in flight, and TLS, which is where most people’s first attempt quietly goes wrong. The good news: part 1 already gave us the thing that carries all of it. A gRPC server is just another service to register on the controller.

Why gRPC, and not just REST

Worth a moment on why we’re reaching for gRPC at all, because for plenty of services a plain JSON-over-HTTP API is the right call and less faff. gRPC earns its place when a few of these matter:

A contract that’s enforced, not hoped for. The .proto is the single source of truth, and both ends are generated from it. You don’t hand-write JSON marshalling, and you don’t find out at runtime that the client and server disagree about a field’s type. Evolve the schema carefully (add fields by number) and old clients keep working.
Clients in any language, for free. The same .proto generates a Go server and a Python, TypeScript, Rust or Java client with nobody writing an SDK by hand. For an internal service that several teams call, that one point can decide it.
It’s built for service-to-service traffic. Binary protobuf is smaller and quicker to encode than JSON, calls multiplex down a single HTTP/2 connection, and streaming (from the client, the server, or both at once) is a first-class thing rather than something you bolt onto REST with websockets.
Deadlines, cancellation and a health protocol come built in, rather than conventions you reinvent for every service.

The trade-offs are real. A browser doesn’t speak gRPC natively, and a binary protocol is fiddlier to poke at than a JSON endpoint you can curl (which is exactly why reflection and grpcurl exist). That’s not a reason to avoid it; it’s the reason this series doesn’t stop at gRPC. In part 4 we put a REST/JSON face on this very service, so the things that call it get the typed, fast core and the things that can’t speak gRPC still get a friendly HTTP surface. You don’t have to pick a side.

Define the contract

gRPC starts with a schema. Here’s a small macguffin service, macguffin.proto:

// proto/macguffin/v1/macguffin.proto

syntax = "proto3";

package macguffin.v1;

option go_package = "gitlab.com/myorg/macguffinsvc/internal/gen/macguffin/v1;macguffinv1";

service MacguffinService {
 rpc GetMacguffin(GetMacguffinRequest) returns (Macguffin);
 rpc ListMacguffins(ListMacguffinsRequest) returns (ListMacguffinsResponse);
 rpc CreateMacguffin(CreateMacguffinRequest) returns (Macguffin);
}

message Macguffin {
 string id = 1;
 string name = 2;
 int32 quantity = 3;
}

message GetMacguffinRequest { string id = 1; }
message ListMacguffinsRequest { int32 page_size = 1; }
message ListMacguffinsResponse { repeated Macguffin macguffins = 1; }
message CreateMacguffinRequest { string name = 1; int32 quantity = 2; }

From proto to Go

If gRPC in Go is new to you, this is the part that catches people out: you don’t write the Go for those messages and that service interface, you generate it from the .proto. The proto is the source of truth; a compiler turns it into Go you import and build against. Same goes for a client in any other language, all from the same file.

That compiler is protoc, and on its own it’s a faff. You install it, then a separate plugin for each output you want (protoc-gen-go for the message types, protoc-gen-go-grpc for the client and server stubs), keep their versions in step, and drive the lot with a command line of -I include paths and --*_out flags that’s easy to get subtly wrong.

buf is the friendlier way to run exactly that. It wraps protoc and its plugins behind a couple of small config files, handles the plugin versions, and turns that gnarly invocation into a single word. It’s become the usual way to work with protobuf in Go, and it’s what we’ll use here.

At a minimum you need three binaries on your PATH: buf itself, and the two plugins it drives. go install is the quickest way to get them:

go install github.com/bufbuild/buf/cmd/buf@latest
go install google.golang.org/protobuf/cmd/protoc-gen-go@latest
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest

Those land in $(go env GOPATH)/bin, so make sure that’s on your $PATH. Then describe what you want generated in a buf.gen.yaml:

# buf.gen.yaml
version: v2
plugins:
 - local: protoc-gen-go
 out: internal/gen
 opt: paths=source_relative
 - local: protoc-gen-go-grpc
 out: internal/gen
 opt: paths=source_relative

Each part of that earns its place. version: v2 is buf’s config format. The plugins list names the generators to run, and we run two, because gRPC in Go arrives in two halves:

protoc-gen-go turns the messages into Go structs, the Macguffin type and the request and response types, in a macguffin.pb.go file.
protoc-gen-go-grpc turns the service into the client and server scaffolding, in a macguffin_grpc.pb.go file, including the MacguffinServiceServer interface you’re about to implement.

out: internal/gen is where the files land, and paths=source_relative lays them out mirroring the proto’s own folders (so proto/macguffin/v1/... becomes internal/gen/macguffin/v1/...) rather than deriving the path from the go_package line. Then run it:

buf generate

Both files appear under internal/gen/macguffin/v1, and we’re ready to write the implementation.

Running that by hand once is fine; remembering to run it every time the .proto changes is where it goes wrong, and the generated code quietly drifts out of step, usually right before a demo. Wire it into go generate instead. Drop a one-line directive in a file at your module root, say gen.go:

// gen.go (at your module root)

package macguffinsvc

//go:generate buf generate

Now go generate ./... regenerates everything from the proto, and it’s the same one command for any other generator you add later. Run it whenever the schema changes, and in CI if you want to catch a stale checkout.

If OpenAPI is your map of the territory

If your mental model of an API contract is an OpenAPI (Swagger) document, a .proto is the same idea wearing fewer clothes: a typed, language-neutral description of a service that both ends generate from. The thing you notice first is how much less of it there is. Here’s that Macguffin message again, and the same shape written as an OpenAPI schema:

message Macguffin {
 string id = 1;
 string name = 2;
 int32 quantity = 3;
}

Macguffin:
 type: object
 properties:
 id:
 type: string
 name:
 type: string
 quantity:
 type: integer
 format: int32

And that pattern holds across the whole service. The proto above, three calls and five messages, is about twenty lines. Describe the same surface in OpenAPI and you’re closer to a hundred, because OpenAPI also pins down the HTTP verbs, paths, status codes and content types: the transport details a proto leaves out on purpose. That isn’t OpenAPI being bloated; it’s describing more. But when the contract is the thing you care about, the proto says it with less ceremony, and it doesn’t wed your API to HTTP, which is exactly what lets us serve this same service over gRPC now and REST later. (We’ll generate a real OpenAPI document from this proto in part 5, for the readers who still want one.)

Implement it

Generating the code gave you the message types and, more to the point, an interface to satisfy. Open macguffin_grpc.pb.go and you’ll find MacguffinServiceServer, one method per RPC in the proto:

// internal/gen/macguffin/v1/macguffin_grpc.pb.go (generated)

type MacguffinServiceServer interface {
	GetMacguffin(context.Context, *GetMacguffinRequest) (*Macguffin, error)
	ListMacguffins(context.Context, *ListMacguffinsRequest) (*ListMacguffinsResponse, error)
	CreateMacguffin(context.Context, *CreateMacguffinRequest) (*Macguffin, error)
	mustEmbedUnimplementedMacguffinServiceServer()
}

That interface is the server-side contract. Each method takes the request message you defined and hands back the response message, plus an error. Writing the type that honours it, the actual logic behind each call, is the part that’s yours: the proto pins down the shape of the conversation, and this is what the service actually does when one happens.

The one curious line is mustEmbedUnimplementedMacguffinServiceServer(). Alongside the interface, buf generated an UnimplementedMacguffinServiceServer struct with a do-nothing stub for every method, and you embed it in your own type. It earns its keep twice over. It satisfies that unexported method, so your type counts as a real implementation. And it future-proofs you: add a new RPC to the proto later and your existing server still compiles, falling back to the stub (which returns a clean “unimplemented” error) until you write the real method.

Before we satisfy that interface, one separation worth making up front. The gRPC server is a delivery mechanism, not the place the data lives. If we stuff the map of macguffins straight inside it and then build an HTTP server next part, we’d have two servers each hoarding their own copy of the same data. So keep the domain, the macguffins and what you can do with them, in its own type, and let each transport be a thin layer over it.

Here’s that domain: an in-memory store standing in for the repository a real service would have. Nothing in it knows about gRPC, HTTP, or any wire format.

// internal/macguffin/store.go

package macguffin

import "sync"

// Macguffin is the domain type. The JSON tags will let a hand-written HTTP
// handler serve it directly in part 3.
type Macguffin struct {
	ID string `json:"id"`
	Name string `json:"name"`
	Quantity int32 `json:"quantity"`
}

type Store struct {
	mu sync.Mutex
	items map[string]Macguffin
	seq int
}

func (s *Store) Get(id string) (Macguffin, bool) {
	s.mu.Lock()
	defer s.mu.Unlock()

	m, ok := s.items[id]

	return m, ok
}

List and Create are the same shape, and NewStore seeds it with a single maltese-falcon. Now the gRPC server is thin: it embeds the stub, holds a store, and each method calls the store and translates the result into the generated protobuf type.

// internal/grpcsvc/server.go

package grpcsvc

import (
	"context"

	"google.golang.org/grpc/codes"
	"google.golang.org/grpc/status"

	macguffinv1 "gitlab.com/myorg/macguffinsvc/internal/gen/macguffin/v1"
	"gitlab.com/myorg/macguffinsvc/internal/macguffin"
)

type Server struct {
	macguffinv1.UnimplementedMacguffinServiceServer

	store *macguffin.Store
}

func New(store *macguffin.Store) *Server {
	return &Server{store: store}
}

func (s *Server) GetMacguffin(_ context.Context, req *macguffinv1.GetMacguffinRequest) (*macguffinv1.Macguffin, error) {
	m, ok := s.store.Get(req.GetId())
	if !ok {
		return nil, status.Errorf(codes.NotFound, "macguffin %q not found", req.GetId())
	}

	return toProto(m), nil
}

// toProto maps the domain type to the generated protobuf DTO.
func toProto(m macguffin.Macguffin) *macguffinv1.Macguffin {
	return &macguffinv1.Macguffin{Id: m.ID, Name: m.Name, Quantity: m.Quantity}
}

ListMacguffins and CreateMacguffin are the same: call the store, map the result. The one habit worth keeping is to return real gRPC status codes (codes.NotFound here) rather than bare errors, so callers get something they can branch on.

That toProto step is worth a second look, because it comes back round later. The domain has one shape, the proto has its own generated Macguffin, so the adapter maps between them. It’s a small price for a single transport. In part 3 we add a second transport with its own generated type and pay that price again, and part 4 is where we stop paying it twice.

Wire it onto the controller

This is the part that earns its keep. First, generate a serve command, the same way the CLI series generated its commands:

gtb generate command \
 --name serve \
 --short "Run the macguffin service"

That scaffolds two files: pkg/cmd/serve/cmd.go (generated, and wired into your command tree for you) and pkg/cmd/serve/main.go, which holds a RunServe function for your logic. Fill it in:

// pkg/cmd/serve/main.go

package serve

import (
	"context"

	"gitlab.com/phpboyscout/go-tool-base/pkg/controls"
	gtbgrpc "gitlab.com/phpboyscout/go-tool-base/pkg/grpc"
	"gitlab.com/phpboyscout/go-tool-base/pkg/props"

	macguffinv1 "gitlab.com/myorg/macguffinsvc/internal/gen/macguffin/v1"
	"gitlab.com/myorg/macguffinsvc/internal/grpcsvc"
	"gitlab.com/myorg/macguffinsvc/internal/macguffin"
)

func RunServe(ctx context.Context, p *props.Props, _ *ServeOptions, _ []string) error {
	controller := controls.NewController(ctx, controls.WithLogger(p.Logger))

	store := macguffin.NewStore()

	grpcSrv, err := gtbgrpc.Register(ctx, "grpc", controller, p.Config, p.Logger)
	if err != nil {
		return err
	}

	macguffinv1.RegisterMacguffinServiceServer(grpcSrv, grpcsvc.New(store))

	controller.Start()
	controller.Wait()

	return nil
}

That’s the whole server. gtbgrpc.Register does four things in one call: it builds a *grpc.Server, wires the standard gRPC health service to the controller’s health reports (the ones we met in part 1), registers Start, Stop and Status against the controller so the lifecycle is handled, and hands you back the server to register your own service on, which is the RegisterMacguffinServiceServer line. After that it’s the same Start() / Wait() we used for the heartbeat.

It reads its port from config (server.grpc.port, falling back to server.port), so a minimal config is:

server:
 grpc:
 port: 50051
 reflection: true

Poke it

Build, run mytool serve, and reach for grpcurl. Reflection is on, so you don’t need the .proto to hand:

$ grpcurl -plaintext localhost:50051 list
grpc.health.v1.Health
grpc.reflection.v1.ServerReflection
grpc.reflection.v1alpha.ServerReflection
macguffin.v1.MacguffinService

$ grpcurl -plaintext localhost:50051 macguffin.v1.MacguffinService/ListMacguffins
{
 "macguffins": [
 { "id": "m-1", "name": "maltese-falcon", "quantity": 1 }
 ]
}

And the health service is already answering, wired straight to the controller, without you registering a thing:

$ grpcurl -plaintext localhost:50051 grpc.health.v1.Health/Check
{ "status": "SERVING" }

That’s the lifecycle work from part 1 paying out: the controller’s health is the gRPC health, and a SIGTERM still drains and stops the server cleanly.

Now turn on TLS

Here’s the bit people brace for. Plaintext gRPC is fine on a laptop and unacceptable the moment it leaves one. With go-tool-base it’s a config change, not a code change.

The fiddly part of local TLS is usually the certificate. A hand-rolled self-signed one means passing a -cacert to every client and clicking past browser warnings. mkcert makes that go away: it creates a local certificate authority and installs it into your system’s (and your browser’s) trust stores, so anything it signs is simply trusted. Set the CA up once:

mkcert -install

Then mint a certificate for the names the service answers on:

mkcert localhost 127.0.0.1 ::1

That writes localhost+2.pem (the certificate) and localhost+2-key.pem (the key), signed by your now-trusted local CA. Doing this properly now pays off later: in part 3 the HTTP server, and in part 5 the API docs in a browser, both lean on that certificate being trusted with no warning.

Point the tool’s config at the pair, under the shared server.tls block:

server:
 grpc:
 port: 50051
 reflection: true
 tls:
 enabled: true
 cert: ./localhost+2.pem
 key: ./localhost+2-key.pem

No code changes. Run mytool serve again and it comes up over TLS:

INFO starting gRPC server tls=true addr=:50051

Because the certificate is signed by a CA your machine already trusts, the client needs no extra flags:

$ grpcurl localhost:50051 macguffin.v1.MacguffinService/GetMacguffin -d '{"id":"m-1"}'
{ "id": "m-1", "name": "maltese-falcon", "quantity": 1 }

A plaintext client is now refused, as it should be. (In production you’d point those same two config keys at whatever your real CA issues; the wiring doesn’t change.) Two details are worth knowing about what just happened, because both are easy to get wrong by hand. The server uses a hardened TLS config (1.2 minimum, AEAD cipher suites, X25519), so you’re not accidentally shipping a weak handshake. And the listener advertises HTTP/2 over ALPN, the h2 protocol gRPC rides on, which sounds like a footnote until you discover that recent gRPC clients refuse a TLS connection that doesn’t offer it. The framework sets it for you; it’s the single most common reason a hand-rolled gRPC-over-TLS server works with old tooling and mysteriously rejects a current client. All of that lives in the shared pkg/tls package.

I put the certificate under server.tls rather than server.grpc.tls deliberately. That shared block is the cert every transport falls back to, so the HTTP server in the next part and the transports after it can use the same one, with a per-transport override only where you actually need it.

The short version

A few files in, you have a real gRPC API: a typed contract, an implementation, health an orchestrator understands, reflection for poking, a clean shutdown, and TLS, and the only part that was actually yours to write was the service logic. The reference for the server helpers is the gRPC component doc, and the add-a-gRPC-service how-to has the manual-wiring path if you ever want it.

Next part puts an HTTP face on the very same controller, REST handlers and the same health endpoints an orchestrator probes, sharing that one certificate.

Building a web service with go-tool-base, part 1: lifecycle and graceful shutdown

Mon, 25 May 2026 00:00:00 +0000

A CLI does its job and gets out of the way. You run it, it prints something or writes a file, the process exits, done. Then one day you want the opposite: a thing that stays running. A server answering requests, a worker chewing through a queue, something that sits there until you tell it to stop. And the moment a process is long-lived, a pile of fiddly problems lands on your desk that a short-lived command never had to think about.

How does it shut down when Kubernetes sends it a SIGTERM, without dropping the requests it’s halfway through? How does anything outside it know it’s alive, or ready for traffic? When one part falls over at 3am, does the whole thing come down, or pick itself back up? None of that is your actual service. It’s the plumbing around it, and it’s the sort of plumbing that’s easy to write almost right and only notice the gap in during an incident.

This is a new series, a companion to the one on building a CLI. That series gets you a working command-line tool; this one turns it into a web service, a piece at a time: gRPC, HTTP, a gateway that bridges the two, TLS across all of them, and live API docs. But every one of those is a long-running process, so we start with the part they all stand on.

Same shape as before, each part stands alone. By the end of this one you’ll have a process that starts cleanly, reports its own health, and shuts down without dropping anything, and you won’t have written the lifecycle code yourself. The series is written against go-tool-base v0.6.0, the release that brings the web-service components in.

What every long-running process needs

go-tool-base’s answer to all of the above is the controls package, and its centrepiece is the Controller. You hand it a set of services, things with a way to start, a way to stop, and a way to report health, and it owns their lifecycle. It starts them, watches for the operating system asking the process to quit, drives a graceful shutdown in the right order, and keeps a running picture of whether everything is alright.

A “service” here is deliberately loose. An HTTP server is one. A gRPC server is one. So is a background worker that wakes every few seconds, or a queue consumer. The controller doesn’t care what’s inside; it cares that it can start it, stop it, and ask after its health. That’s the whole trick: get those three verbs right once, in one place, and everything you bolt on later inherits them.

A service in thirty lines

Let’s build the smallest useful one: a heartbeat that logs a tick every second. It isn’t exciting, but it’s a real long-running service, and it shows every moving part without a transport getting in the way.

// main.go

package main

import (
	"context"
	"os"
	"sync/atomic"
	"time"

	"gitlab.com/phpboyscout/go-tool-base/pkg/controls"
	"gitlab.com/phpboyscout/go-tool-base/pkg/logger"
)

func main() {
	log := logger.NewCharm(os.Stderr, logger.WithTimestamp(true))

	// The controller owns the process lifecycle: it starts registered services,
	// watches for SIGINT/SIGTERM, and drives a graceful shutdown.
	controller := controls.NewController(context.Background(), controls.WithLogger(log))

	var beats atomic.Int64

	controller.Register("heartbeat",
		// Start launches the work. The context is cancelled when the controller
		// shuts down, so the goroutine just watches ctx.Done().
		controls.WithStart(func(ctx context.Context) error {
			go func() {
				ticker := time.NewTicker(time.Second)
				defer ticker.Stop()

				for {
					select {
					case <-ctx.Done():
						return
					case <-ticker.C:
						log.Info("beat", "count", beats.Add(1))
					}
				}
			}()

			return nil
		}),
		// Stop runs during shutdown for any explicit cleanup.
		controls.WithStop(func(_ context.Context) {
			log.Info("heartbeat stopping", "total_beats", beats.Load())
		}),
		// Status reports health. Here we're healthy as long as we're ticking.
		controls.WithStatus(func() error { return nil }),
	)

	controller.Start()
	controller.Wait()
}

A few things are earning their keep there. NewController takes a context and some options, here just a logger. Register names a service and gives it its three verbs through functional options: WithStart launches it (and is handed a context that gets cancelled when the controller shuts down, which is the hook the goroutine watches), WithStop is called during shutdown for cleanup, and WithStatus answers “are you alright?”. Then controller.Start() launches everything and controller.Wait() blocks until the whole thing has stopped. The three options are all in pkg/controls.

Build it and run it:

go run .

INFO beat count=1
INFO beat count=2
INFO beat count=3

A service, running. Now for the half that’s easy to get wrong.

Shutting down on purpose

Press Ctrl-C, or send the process a SIGTERM the way an orchestrator would, and watch what it does:

WARN [Controller] : received signal signal=terminated
WARN [Controller] : Stopping Services
INFO heartbeat stopping total_beats=3
INFO [Controller] : Stopped

Nothing in our thirty lines handled a signal. The controller registered its own handlers for SIGINT and SIGTERM, and when one arrived it cancelled the context that every service’s Start is watching, gave them a window to finish, ran each Stop, and exited cleanly. That cancel-the-context-then-Stop order is the thing: it’s exactly what stops an HTTP server from dropping requests it’s mid-way through when the pod rolls. We’ve got it here for a heartbeat that does nothing on the way out, and we’ll get the same order for free for every real transport we add later.

You can tune the window with WithShutdownTimeout, and turn the signal handling off entirely (handy in tests) with WithoutSignals. The defaults are the right ones for a service in a container.

Health, before anything’s asking

That third verb, WithStatus, is the start of the health story, and it’s worth seeing now even though nothing’s calling it yet. The controller can report three separate things: overall status, liveness, and readiness. Those aren’t the same question. Liveness is “is this process wedged and in need of a restart”; readiness is “should traffic come here yet”. An orchestrator uses them differently, which is why they’re kept apart, and a service can answer them separately by adding WithLiveness and WithReadiness alongside WithStatus.

Right now nothing asks, because we’ve no transport. But this is the quiet payoff of putting lifecycle first: when we add an HTTP server in part 3, these reports are what back its /healthz, /livez and /readyz endpoints, and when we add gRPC in part 2 they back the standard gRPC health service, with no re-plumbing on our side. The controller also carries a restart policy for services that should pick themselves back up, and standalone health checks for things like “can I still reach the database”, but those earn their place once we’ve something worth checking.

Where this leaves us

A few lines in, we’ve a process that starts, ticks along, answers for its own health, and stops cleanly when the platform asks it to, on a controller that the real transports will register against unchanged. The heartbeat is a stand-in. Next part we swap it for a proper gRPC service, give it TLS, and the controller barely notices the difference, which is the entire point of it.

If you want to read ahead, the controls component has the full interface, and the service-orchestration deep-dive covers how the startup ordering and shutdown actually work underneath.

Building a CLI with go-tool-base, part 5: a CLI that updates itself

Sun, 24 May 2026 00:00:00 +0000

You ship version one. A week later someone finds a bug, you fix it, you cut version two. Now for the awkward part: how does the person who installed version one ever get version two? Email them? Hope they wander back to the install page? For a CLI that lives on people’s machines, “go and re-download it” is the answer that quietly strands half your users on old, broken builds. This part closes that gap, and like most of this series, the work is already done for you: your tool has shipped with an update command since part 1.

As before, this is written against go-tool-base v0.6.0 (gtb version).

The command is already there

update is one of the default features, so it’s been in your binary all along. Your users run:

mytool update

and the tool fetches the newest release, checks it, and replaces itself in place. No package manager, no re-download, no instructions. The rest of this part is about what that one command actually does, and how to make sure the binary it pulls down is the one you shipped.

Where it looks for releases

A tool can’t update itself without knowing where its releases live. That’s the --repo you passed back in part 1: it filled in your tool’s release source, the platform, owner and repository it checks. For --repo myorg/mytool that’s github.com/myorg/mytool, and mytool update looks at that project’s releases.

go-tool-base speaks more than one platform here, GitHub, GitLab, Gitea, Codeberg, Bitbucket, or a plain HTTP server, so the same command works whether you publish on github.com or your own GitLab. If you ever need to point somewhere else (a mirror, a private host), the custom release source how-to covers it; for a private repository it reads a token the same way the rest of the tool does.

What `update` does, step by step

When a user runs it, the command walks a short, careful path:

Resolve the latest release from your release source.
Compare versions. It reads the version baked into the running binary and compares it, as semver, against the latest. If you’re already current, it says so and stops: already running latest version, v1.2.0. (If your build somehow reports a version ahead of the latest published, it tells you off in character: your tardis travelled too far into the future...)
Download the right archive for the user’s OS and architecture.
Verify it before trusting it (the next section).
Replace the running binary with the new one, in place.
Bring the config along. If your tool has the init feature (it does by default), the update then runs the new binary’s init over the user’s config directory to fold in anything the release added.

That last step is easy to miss and matters more than it looks. A new version often ships new config: a key for a feature you just added, a changed default. Rather than leave the user a version behind, with code that expects settings their config file has never heard of, update re-runs init against their existing config once the swap is done, non-interactively (it passes --skip-login --skip-key, so nobody gets re-prompted for a token). It’s the same initialiser system from part 2, reused: the merge keeps what the user set and adds what the new version introduced, so the binary and its config move forward together. Turn the init feature off and this step is simply skipped, there’s no config to keep in step with.

There are two flags worth knowing. --version v1.3.0 targets a specific release instead of the latest, handy for pinning or rolling back. And --force updates even when the version check thinks you don’t need to. Most of the time, a bare mytool update is the whole story.

Downloaded isn’t the same as trusted

A binary that arrives over the network is a binary you didn’t build on the machine it’s running on, and a self-updater that swaps itself for whatever the server sent is a lovely way to ship a corrupted or tampered build straight into your users' hands. So before the swap, update verifies what it downloaded against a checksum manifest, the checksums.txt GoReleaser produces alongside your binaries. If the hash of the downloaded archive doesn’t match the one in the manifest, the update aborts and nothing gets replaced.

By default this is best-effort: a release that ships a checksums.txt is verified, but a release without one is updated with a warning rather than a hard stop. When you want the guarantee, make it mandatory:

# in your tool's config
update:
 require_checksum: true

Now a missing or mismatched checksum is a refusal, not a shrug. I wrote up why this matters, and exactly what it does and doesn’t buy you, in verifying your own downloads.

The limit is worth stating plainly, because it’s the whole reason there’s a “part two” to this story. A checksum proves the binary matches the manifest on the same release page. It catches a corrupted download or a botched upload cold. What it cannot catch is an attacker who owns the release platform and swaps both the binary and its checksum in the same breath, because then the two still agree. Closing that gap needs a signature whose trust root the release host can’t reach, which is a different piece of machinery (and a post of its own). go-tool-base now does exactly that: self-update signature verification has shipped, the binary checking a detached signature against a key it both embeds and fetches over WKD (how it works). Until you turn signing on for your own tool, checksums are the floor, and a worthwhile one.

Seeing it work without publishing anything

Here’s the catch with writing about self-update: you can’t update from a release you haven’t published, and your tutorial tool isn’t on anyone’s GitHub. There’s a flag for exactly this, meant for offline and air-gapped installs but perfect for a look under the hood: --from-file installs from a local release archive instead of the network.

Build a snapshot of your tool the way your release pipeline would (GoReleaser’s --snapshot builds the archives without publishing), then point update at one:

goreleaser release --snapshot --clean
mytool update --from-file ./dist/mytool_Linux_x86_64.tar.gz

You’ll watch the same extract-and-swap the network path uses, with nothing published and no release source involved. It’s also genuinely useful in its own right, for shipping into environments that can’t reach the internet.

The real loop

In production the cycle is the one part 1 already set you up for. The project gtb scaffolds ships a GoReleaser config and a release pipeline, so the flow is:

Tag a version and push the tag.
CI builds the binaries for every OS and architecture, generates checksums.txt, and publishes them as a release on your source.
Your users run mytool update and get it, verified.

You write git tag v1.3.0 && git push --tags; everyone who installed v1.2.0 is one command away from the fix. That’s the whole point of putting the update channel inside the tool: shipping a fix becomes tagging a release, and nothing else.

What this buys you

A tool that updates itself turns “please go and reinstall” into mytool update, and a tool that verifies what it updates to turns “I hope that download was clean” into a checked guarantee. Both came with the scaffold; the only work was understanding them. The full reference, including the config keys and the per-platform release sources, is in the update command docs and the auto-update concepts page.

Next part is the last one, and it’s about what happens after your tool is out there doing its job: telemetry and logging, so you can see how it’s actually being used without spying on the people using it. Until then, tag a release and watch your tool catch up to itself.

Building a CLI with go-tool-base, part 4: an AI dungeon master

Sat, 23 May 2026 00:00:00 +0000

I run a Dungeons & Dragons game on the odd weekend, so when I sat down to put an AI feature inside a CLI, my first instinct wasn’t a chatbot. It was: could the tool run a little adventure, with an AI as the dungeon master? It turns out that’s a near-perfect way to learn the chat client, because the thing that makes a game trustworthy, rules the players can’t break, is exactly the thing that makes any AI feature trustworthy. So this part builds mytool adventure: a tiny dungeon you play in your terminal, narrated by an AI that is firmly on a leash.

Part 3 pointed AI at your CLI from the outside (an agent driving your commands over MCP). This part goes the other way: AI inside your tool, as a feature you write. The worry everyone has about that is fair, AI output is unpredictable, and a CLI is meant to be dependable. The whole lesson here is how you square those two: you don’t hope the model behaves, you box it in with rules it can’t escape and mechanics it doesn’t get to invent.

As before, this is written against go-tool-base v0.6.0 (gtb version).

Behind the DM screen

A turn of our game looks like this: the player types what they want to do, the AI dungeon master narrates what happens and offers a few choices, and round it goes until the adventure reaches an end. The trick is where the truth lives. The model’s job is the prose, and only the prose. Everything else is yours:

The rules live in the system prompt: what the DM may and may not do.
The mechanics live in Go functions the model calls as tools (dice, combat). It never makes a number up.
The state lives in a Go struct you hand the model fresh every turn, so it never has to remember, and can’t quietly rewrite history.
The shape of each turn is a typed Go struct the model fills in, so your code always gets back something it can render, never a wall of prose to parse.

Two go-tool-base capabilities do the heavy lifting: the AI calling your Go functions, and the AI handing back a typed struct instead of text you have to regex. The game is just a fun excuse to use both at once.

Wiring a provider

The chat client (pkg/chat) is a library you import; you don’t need any special feature flag for it. It does need an API key, and it’ll find one from a few places. The simplest, for now, is the well-known environment variable for your provider:

export ANTHROPIC_API_KEY="sk-ant-..." # or GEMINI_API_KEY, OPENAI_API_KEY

That’s the bottom of the client’s lookup chain, which is fine for playing locally. For a tool you actually ship, go-tool-base has the ai feature and its mytool init wizard (the same initialiser system from part 2) to store the key properly, and there’s a whole post on where a CLI should keep your keys. For learning the client, an env var is plenty.

Scaffold the command

You know this step from part 1:

gtb generate command --name adventure --short "Play a dungeon adventure"

Everything below goes in the RunAdventure function the generator left you in pkg/cmd/adventure/main.go, plus a couple of types and helpers in the same package.

The state is yours, not the model’s

Start with the truth. The game state is a plain Go struct that you own. The model never holds it; instead you hand it the current state at the top of every turn (more on that in the loop). This is the part to grow: start small, then add rooms, items, NPCs, quest flags, whatever your adventure needs. Nothing else in the design has to change when you do.

// GameState is the single source of truth for the game. Extend it freely.
type GameState struct {
	PlayerHP int
	Location string
	Inventory []string
	Foes map[string]int // foe name -> remaining hit points
}

// summary renders the state into a line the model is given each turn.
func (g *GameState) summary() string {
	foes := make([]string, 0, len(g.Foes))
	for name, hp := range g.Foes {
		foes = append(foes, fmt.Sprintf("%s (%d HP)", name, hp))
	}
	return fmt.Sprintf("You have %d HP, at %s, carrying %s. Foes: %s.",
		g.PlayerHP, g.Location, strings.Join(g.Inventory, ", "), strings.Join(foes, ", "))
}

And the shape of a turn, the thing the model has to produce:

type Turn struct {
	Narration string `json:"narration"`
	Choices []string `json:"choices"`
	GameOver bool `json:"game_over"`
}

The dungeon master’s tools

A tool in pkg/chat is a chat.Tool: a name, a description the model reads to decide when to use it, a parameter schema, and a handler. The handler gets the model’s arguments as raw JSON and returns any value (which the framework JSON-encodes back to the model) or an error.

The simplest possible one is a die roll. This is the canonical “give the model something it’s bad at” tool, because language models cannot be trusted to roll fairly or even add up:

func rollTool() chat.Tool {
	return chat.Tool{
		Name: "roll",
		Description: "Roll a die with the given number of sides; returns 1..sides.",
		// Use an anonymous struct so the schema's properties sit at the top level,
		// which is where SetTools looks. A named type would hide them behind a $ref.
		Parameters: jsonschema.Reflect(struct {
			Sides int `json:"sides" jsonschema:"description=number of sides on the die"`
		}{}),
		Handler: func(_ context.Context, args json.RawMessage) (any, error) {
			var a struct {
				Sides int `json:"sides"`
			}
			if err := json.Unmarshal(args, &a); err != nil {
				return nil, err
			}
			if a.Sides <= 0 {
				a.Sides = 20
			}
			return rand.Intn(a.Sides) + 1, nil
		},
	}
}

That comment about the anonymous struct matters, by the way. Reflect a named type and jsonschema emits a top-level reference with the real fields tucked inside, and the tool ships with no parameters at all. An anonymous struct inlines them where the framework expects. It’s the one sharp edge in the whole exercise.

Combat is where state actually changes, so combat is a tool too. Note it takes the foe by name and looks it up in Foes, so it works for the goblin and for any creature you add later, without touching this function:

func attackTool(game *GameState) chat.Tool {
	return chat.Tool{
		Name: "attack",
		Description: "Resolve the player's attack on a named foe. Rolls to hit, applies damage.",
		Parameters: jsonschema.Reflect(struct {
			Target string `json:"target" jsonschema:"description=the name of the foe being attacked"`
		}{}),
		Handler: func(_ context.Context, args json.RawMessage) (any, error) {
			var a struct {
				Target string `json:"target"`
			}
			if err := json.Unmarshal(args, &a); err != nil {
				return nil, err
			}
			hp, ok := game.Foes[a.Target]
			if !ok {
				return map[string]any{"error": "no such foe: " + a.Target}, nil
			}
			if rand.Intn(20)+1 < 10 {
				return map[string]any{"hit": false, "foe": a.Target}, nil
			}
			dmg := rand.Intn(6) + 1
			hp -= dmg
			if hp < 0 {
				hp = 0
			}
			game.Foes[a.Target] = hp
			return map[string]any{
				"hit": true, "foe": a.Target, "damage": dmg,
				"foe_hp": hp, "defeated": hp == 0,
			}, nil
		},
	}
}

A bad target comes back as a plain error string, which the framework hands to the model so it can recover (apologise, pick a real foe) rather than crash.

That’s the whole tool set, and there’s deliberately nothing here for reading the state. The model never fetches it. Instead the loop hands it the current state at the top of every turn, which we wire up shortly. A language model has no memory you can rely on, so rather than trust it to remember the fight, you give it the truth each time.

The turn is a tool too

Here’s the neat part. The chat client won’t let a single call both run tools and return a typed struct, they’re separate modes. So instead of asking for the struct afterwards, we make submitting the turn into a tool of its own. The dungeon master ends its turn by calling submit_turn, and its handler captures the typed Turn into a variable we hold:

func submitTurnTool(out *Turn) chat.Tool {
	return chat.Tool{
		Name: "submit_turn",
		Description: "End your turn. Call this exactly once, last, with the turn's outcome.",
		Parameters: jsonschema.Reflect(struct {
			Narration string `json:"narration" jsonschema:"description=two-sentence narration of what just happened"`
			Choices []string `json:"choices" jsonschema:"description=the actions the player may take next"`
			GameOver bool `json:"game_over" jsonschema:"description=true only if the game has ended"`
		}{}),
		Handler: func(_ context.Context, args json.RawMessage) (any, error) {
			if err := json.Unmarshal(args, out); err != nil {
				return nil, err
			}
			return "turn recorded", nil
		},
	}
}

So the turn’s structure is enforced by a schema, same as any other tool’s parameters. Your loop gets a populated Turn every round, never prose.

The rules

This is where you bound the model. The system prompt is the rulebook, and it leans hard on the tools so the DM has no room to freelance the mechanics:

const dmRules = `You are the dungeon master of a short fantasy adventure. Each turn
you are given the current game state and the player's action.

Resolve the action and end the turn:
- If the player attacks, you MUST call the attack tool with the foe's name to
 resolve it. Do not decide the hit or the damage yourself.
- For any other chance event, call the roll tool and use its result.
- For simple actions, just narrate them.
- Then call submit_turn exactly once: a two-sentence narration, two or three
 choices, and game_over.

Trust the state you are given; never contradict it. A foe at 0 hit points is dead
and stays dead. The game ends when the player's hit points reach 0 (they lose), or
when the player reaches a satisfying ending. When it ends, set game_over and narrate
the finish.

Keep the tone light and quick.`

Two of those lines carry the weight. Trusting the state you are given, and never contradicting it, is what keeps the world consistent: the state is handed in fresh every turn (the next section), so the model works from the truth instead of from a memory it does not reliably have. And you MUST call the attack tool is what stops it quietly deciding hits and damage itself when it would rather just narrate. Those two are the difference between a game with rules and a model telling a story.

The loop

Now stitch it together. Create the client with the rules as its system prompt, register the tools once, and run a turn each time the player acts:

func RunAdventure(ctx context.Context, props *props.Props, opts *AdventureOptions, args []string) error {
	game := &GameState{
		PlayerHP: 20,
		Location: "the mouth of a damp cave",
		Inventory: []string{"a short sword", "a guttering torch"},
		Foes: map[string]int{"goblin": 12},
	}
	var turn Turn

	client, err := chat.New(ctx, props, chat.Config{
		SystemPrompt: dmRules,
		MaxSteps: 8, // roll/attack, then submit_turn
	})
	if err != nil {
		return err
	}

	if err := client.SetTools([]chat.Tool{
		rollTool(),
		attackTool(game),
		submitTurnTool(&turn),
	}); err != nil {
		return err
	}

	action := "I step into the cave."
	for {
		turn = Turn{}
		// Hand the model the current truth, then the player's action.
		input := fmt.Sprintf("State: %s\nThe player: %s", game.summary(), action)
		if _, err := client.Chat(ctx, input); err != nil {
			return err
		}

		fmt.Println("\n" + turn.Narration)
		if turn.GameOver {
			return nil
		}
		action, err = chooseAction(turn.Choices)
		if err != nil {
			return err
		}
	}
}

The same client runs every turn, so the conversation and the tools carry through the whole game; and the State: line you prepend is always current, because the attack tool mutated game last turn. The model is never trusted to remember, only to narrate.

The one helper I glossed is chooseAction. A bare fmt.Scanln would do, but we can do much better with almost no effort, and make a point while we’re at it. The framework already leans on Charm’s huh for its init wizard, you met it in part 2, so we’ll use the same library for a proper menu, with one deliberate addition:

func chooseAction(choices []string) (string, error) {
	const other = "__other__"

	opts := make([]huh.Option[string], 0, len(choices)+1)
	for _, c := range choices {
		opts = append(opts, huh.NewOption(c, c))
	}
	opts = append(opts, huh.NewOption("Something else...", other))

	var pick, custom string
	form := huh.NewForm(
		huh.NewGroup(
			huh.NewSelect[string]().
				Title("What do you do?").
				Options(opts...).
				Value(&pick),
		),
		// A second step that only appears when the player chose "Something else".
		huh.NewGroup(
			huh.NewInput().
				Title("Describe your action").
				Value(&custom),
		).WithHideFunc(func() bool { return pick != other }),
	)
	if err := form.Run(); err != nil {
		return "", err
	}

	if pick == other {
		return custom, nil
	}
	return pick, nil
}

The select gives the player a tidy arrow-key menu instead of typing a number, but the addition that earns its keep is the last option. “Something else…” is always there, and choosing it unfolds a second step (huh shows or hides a group with WithHideFunc) where the player types whatever they actually want to do. That free text goes straight to the dungeon master as the next turn’s input, and because the DM is an AI bound by the rules rather than a switch statement over three fixed choices, it just copes. Bargain with the goblin, search your pockets, set the cave alight: the model narrates it within the rules you gave it, rolling and applying damage through the same tools. That is the agency a scripted game can’t offer, and it’s the natural place to start building your own richer interactivity on top.

Play it

Set your key, build, and go:

export ANTHROPIC_API_KEY="sk-ant-..."
just build
./bin/mytool adventure

A turn looks like this (your wording will differ every time; the mechanics won’t):

You swing your short sword at the goblin, the blade whistling through the damp cave
air. The creature snarls as it tries to dodge your blow.

What do you do?
> Attack the goblin again
 Try to push deeper into the cave
 Retreat to the entrance
 Something else...

Your blade whistles through the air, but the nimble goblin dances back just in
time. It lunges forward with a rusty dagger in return, yet its clumsy strike only
finds empty air.

What do you do?
> Swing your sword again!
 Try to intimidate the creature
 Retreat from the cave
 Something else...

Behind that, the dungeon master called attack each turn (a hit, then a miss), the goblin’s hit points changed in the GameState you own, and the next turn handed that updated state straight back to the model. The prose is the model’s; every number is yours.

The pattern under the game

Strip the dungeon away and you’re left with the thing worth keeping. An AI feature you can ship is one where you’ve kept the model away from everything that has to be right: the rules live in the system prompt, the mechanics in typed Go tools the model must call, the state in a struct you hand it fresh each turn rather than trust it to remember, and the output in a struct it fills in rather than free text. Do that and the model’s unpredictability is confined to exactly where you want it, the wording, and walled out of everywhere you don’t, the maths, the state, the shape of the result.

Two honest limits worth knowing. There’s no temperature dial on the client (the setting that would let you turn the model’s randomness down), so you can’t make the prose reproducible; you make the mechanics reproducible instead, which for most features is what you actually needed. And a tool calling loop is several round-trips to the model per turn, so it’s not free, keep MaxSteps tight for anything interactive.

That’s the foundation, and the state struct is already sized for more than one fight: it carries a location, an inventory and a map of foes you’ve barely touched. Add a move tool that updates Location, a use_item tool that reaches into Inventory, a second creature in Foes, even a give_quest flag, and the adventure grows without the architecture changing. The model just gets more tools to call and more truth to read. Saved games come nearly free, too: the client can snapshot and resume a conversation. Next part leaves AI behind and gets the tool ready to look after itself: shipping signed self-updates, so a new release reaches your users safely. Until then, go explore the cave.

Building a CLI with go-tool-base, part 3: expose your CLI to AI agents

Fri, 22 May 2026 00:00:00 +0000

“Make it work with AI” is the request that lands on your desk with a thud and no further detail. The first time it landed on mine I braced for a treadmill of integration work: an adapter for this assistant, a wrapper for that one, one per client, forever. Then I looked at the hello command we built back in part 1. It has a name, a one-line description, and (once you give it some) typed, documented flags. That is exactly the shape an AI agent needs to call a tool. You already did the hard part.

This part wires that up: turning the CLI you’ve been building into something an AI assistant can drive, with no AI code of your own. The how-it-works behind it is in your CLI is already an AI tool; here we just use it.

A version note, as in the earlier parts: this is written against go-tool-base v0.6.0 (gtb version). The tool is young and moving, so if you’re on a newer release and a command or its output has shifted, check there first.

The translator you already have

An AI agent that wants to call local tools needs three things: a list of named operations, a description of each so it knows when to reach for them, and a typed parameter schema for each so it knows how to call them. A good CLI is already all three. The only missing piece is a translator between “this is a CLI” and “this is a set of tools an AI can call”, and that translator is the Model Context Protocol (MCP), an open standard every serious assistant now speaks.

Your tool already ships it. mcp is one of the default features, so it’s been in your binary since you scaffolded in part 1, no flag required. Check:

./bin/mytool mcp --help

You’ll see subcommands you never wrote. The rest of this part is just three of them.

See what the agent sees

Before you connect anything, look at what your tool would expose. This writes the tool definitions to a file:

./bin/mytool mcp tools

{
 "tools": [
 {
 "name": "mytool_hello",
 "description": "Say hello",
 "inputSchema": {
 "type": "object",
 "properties": {}
 }
 }
 ]
}

That’s your hello command, seen from an agent’s side of the glass. The name is your tool’s name and the command path joined with an underscore; the description is the Short you gave it in part 1; the inputSchema is empty because hello has no flags yet. Add a flag and it shows up here as a property, with the type and help text you already wrote. There’s no second schema to keep in sync, because the command tree is the schema.

A few things are deliberately left off this list: hidden and deprecated commands, pure command groups that don’t do anything themselves, and the mcp, help and completion plumbing. So mcp tools doubles as an audit: it’s exactly what an agent can reach, and nothing else.

Run the server

One command turns the whole thing on:

./bin/mytool mcp start

It doesn’t print a banner and march off doing things. It sits quietly, speaking MCP as JSON-RPC over standard input and output, waiting for an assistant to talk to it. You won’t run this by hand much; the assistant launches it for you. But it’s worth knowing what happens when the agent calls one of your tools: the server re-runs your own binary with the arguments the agent supplied, captures the output, and hands it back. The agent isn’t poking at your internals. It’s running mytool hello, the same command a human would, and getting the same result.

Point an assistant at it

The quickest way is to let the tool write the client config for you. For Claude Desktop:

./bin/mytool mcp claude enable

There are cursor and vscode variants too. Restart the assistant and your CLI is in its toolbox.

If you’d rather wire it by hand (or your client isn’t one of those three), the config is small. Point the client at your binary with mcp start as its arguments, using the absolute path:

{
 "mcpServers": {
 "mytool": {
 "command": "/absolute/path/to/bin/mytool",
 "args": ["mcp", "start"]
 }
 }
}

Claude Desktop keeps that in claude_desktop_config.json (under ~/Library/Application Support/Claude/ on macOS, %APPDATA%\Claude\ on Windows); Cursor uses ~/.cursor/mcp.json; VS Code’s Copilot reads github.copilot.mcpServers in your settings. The shape is the same everywhere. From here, ask the assistant to say hello and watch it call mytool_hello.

The agent’s reach is exactly your CLI’s reach

This is the part worth being calm about. Exposing your CLI over MCP doesn’t widen its surface by an inch. The agent can call the commands you shipped, with the parameters you defined, and nothing else. It can’t invent a command or pass a flag you never wrote. The boundary of what it can do is the boundary you drew when you built the tool, and mcp tools shows you that boundary in full. If there’s a command you don’t want an agent touching, mark it hidden and it drops off the list.

For a long-running or remote setup there’s also ./bin/mytool mcp stream, which serves the same tools over HTTP instead of stdio; the mcp reference has the details. For most desktop assistants, start over stdio is all you need.

What it comes down to

You turned the CLI you’ve been building into an agent-callable tool with one command and zero lines of AI code, because the real work, naming your operations and documenting their inputs, you finished the moment your --help was any good. Every command you add from here is a new tool the agent gets for free.

Next part goes the other way: instead of letting an assistant drive your tool from outside, we put AI inside it, wiring up a provider and building a feature against go-tool-base’s chat SDK. Until then, add a command or two and watch them appear in mcp tools. The agent’s toolbox grows as your CLI does.

Building a CLI with go-tool-base, part 2: configuration your tool can trust

Thu, 21 May 2026 00:00:00 +0000

In part 1 you scaffolded a tool and gave it a hello command. It says the same thing every time, which is fine for a first command and useless for a real one. The moment a tool does anything worth doing it needs settings: an endpoint, a default, a token, a log level. And the moment you have settings, you have the problem nobody warns you about. You set one in a file, the tool ignores it, the code that reads it looks perfectly correct, and an hour later you find you’d typed tiemout. Nothing in the whole stack thought that worth a word.

The good news is you don’t have to build any of this. Your scaffold already wired up a config system in part 1, the same one the rest of go-tool-base uses. This part puts it to work: where a setting’s value actually comes from, how to ship sensible defaults alongside the command they belong to, how to layer files so a team and a laptop can disagree politely, and how to turn a fat-fingered key from a silent shrug into an error that tells you exactly what you got wrong.

The same version note as part 1, since each of these stands on its own: everything here is written against go-tool-base v0.6.0 (gtb version will tell you what you’re on). The tool is young and still changing shape, so if you’re on a newer release and a detail has drifted, that’s the first thing to check. I’ll flag anything that breaks across versions as it comes up.

You already have a config system

The root command loads configuration for you before any of your command code runs, merges every source together, and hands the result to each command through Props. By the time your RunHello runs, props.Config is populated and ready.

A value can arrive from several places at once, so there’s an order. Highest wins:

Command-line flags
Environment variables (your tool’s prefix plus the key, so hello.greeting reads MYTOOL_HELLO_GREETING, with the dots turned into underscores)
Config files (on disk, in the order they were loaded)

That ladder is the mental model for what beats what: a flag beats an env var, an env var beats a file. The files are worth pinning down, though, because there’s more than one and they don’t all come from the same place. This is the bit that’s easy to trip over:

Embedded defaults are baked into the binary, one slice per command. You don’t read these at runtime directly. The init command (coming up) bakes them into your config file for you.
The file init writes, ~/.mytool/config.yaml, is the default the tool reads, along with a machine-wide /etc/mytool/config.yaml if one exists.
Files passed with --config replace those defaults for that run rather than adding to them. Name one or more and the tool reads exactly those.

We’ll set each of these up in turn. The full reference lives in the config docs.

Reading a value is one call, and it’s typed:

greeting := props.Config.GetString("hello.greeting")
timeout := props.Config.GetDuration("server.timeout")
debug := props.Config.GetBool("verbose")

Give a command a setting

Let’s make hello configurable. Open pkg/cmd/hello/main.go (your file, the one the generator leaves alone) and read the greeting from config instead of hard-coding it:

func RunHello(ctx context.Context, props *props.Props, opts *HelloOptions, args []string) error {
	greeting := props.Config.GetString("hello.greeting")
	props.Logger.Info(greeting)
	return nil
}

Build and run it:

just build
./bin/mytool hello

ERRO failed to load config: no configuration files found
please run init, or provide a config file using the --config flag

Not what you expected, maybe, but it’s the right instinct from the tool. It has no configuration to read yet, and rather than guess, it stops and says so. Which brings us neatly to where settings actually come from.

Defaults belong to the command

You could drop a default into the project’s central config, and for something truly global like the log level that’s the right home. But a setting that belongs to hello should live with hello, not in a shared file you have to remember to edit every time you add a command. The generator does this for you, you just have to ask. Back in part 1 you generated hello without config support, so run the same command again with --assets:

gtb generate command --name hello --short "Say hello" --assets

This is safe to re-run. The generator honours the code you’ve already written: it refreshes the boilerplate cmd.go, adds the asset scaffolding, and leaves your main.go, and the RunHello you’ve been editing, completely alone. One thing to hold off on here: don’t reach for --force. Force rewrites everything, including that main.go, which is exactly the work you want to keep.

You now have pkg/cmd/hello/assets/init/config.yaml, and the generator has already opened it under the command’s own namespace:

hello:

Fill in your defaults under it:

hello:
 greeting: Hello
 style: plain

Those values are embedded into the binary as an asset, and the generated cmd.go registers them with Props for you (props.Assets.Register("hello", &assets)), so the config system knows where your command’s defaults live. A quick word on style, since we’ll lean on it shortly: it’s a second setting I’m giving a default now so it’s ready when we need it. Plain says the greeting as written; loud will shout it.

That per-command home comes with one rule worth taking seriously: namespace your keys. Notice the generator opened the file under a hello: key rather than at the top level. Copy that. Every command ships its defaults in its own embedded file, and those files are all merged together to build the config, but the order they merge in is not guaranteed. If two commands both defined a top-level timeout, which one won would be a toss-up that could flip between builds. Keep each command’s settings under its own name (hello.greeting, report.timeout) and the clash can’t happen in the first place. The generator namespacing the file for you is a hint worth taking.

One thing the defaults file does not do is set values through struct tags. If you later add a default:"info" tag to a config field, that’s documentation for the error messages, nothing more. Real defaults live here, in the embedded YAML. It’s an easy thing to assume otherwise and then wonder why your default never applied.

First run: init

So your defaults are baked into the binary. The tool still needs an actual config file to read, and that’s what init is for. It’s one of the features your tool shipped with, so it’s already there:

./bin/mytool init

INFO Configuration initialised in /home/you/.mytool/config.yaml

Open that file and you’ll find your command’s defaults waiting in it, merged with the framework’s own:

hello:
 greeting: Hello
 style: plain
log:
 level: info

That’s the missing piece. init gathers every command’s embedded defaults through the Assets layer, writes them to ~/.mytool/config.yaml, locks the file down to 0600 (it may hold secrets later), and drops in a .gitignore so nobody commits it by accident. Now hello has something to read:

Prefer no init step? init is a feature, and you can leave it out of your tool’s feature set. With it off, the tool loads its embedded defaults directly and runs with no config file at all, you’d only add one to override something. That suits a small, self-contained tool. This tutorial keeps init on, which is the default and the right call while a tool is finding its feet, so the rest of the article assumes it.

./bin/mytool hello

INFO Hello

Setup that needs a human: initialisers

Static defaults cover the values you can decide for the user. Some you can’t: a token, an API key, an endpoint that differs per person. Writing a blank or guessed value for those is worse than useless. This is where go-tool-base does something I’ve not seen many CLI frameworks bother with: it lets a command bring its own first-run setup, and wires it in for you. It’s one of the genuine reasons to build on the framework rather than roll your own, so it’s worth a proper look.

Generate a command with --with-initializer:

gtb generate command --name greet --short "Greet someone" --with-initializer

Alongside the usual files you get an init.go. It’s generated and marked DO NOT EDIT, and it does all the wiring. Here’s the heart of it:

// Code generated by gtb. DO NOT EDIT.
package greet

func init() {
	setup.Register(props.FeatureCmd("greet"),
		[]setup.InitialiserProvider{func(p *props.Props) setup.Initialiser {
			if skipGreet {
				return nil
			}
			return &GreetInitialiser{}
		}},
		[]setup.SubcommandProvider{func(p *props.Props) []*cobra.Command {
			return []*cobra.Command{NewCmdInitGreet(p)}
		}},
		[]setup.FeatureFlag{func(cmd *cobra.Command) {
			cmd.Flags().BoolVar(&skipGreet, "skip-greet", false, "skip initializing greet configuration")
		}},
	)
}

type GreetInitialiser struct{}

func (i *GreetInitialiser) Name() string { return "greet" }

func (i *GreetInitialiser) IsConfigured(cfg config.Containable) bool {
	return cfg.IsSet("greet")
}

func (i *GreetInitialiser) Configure(p *props.Props, cfg config.Containable) error {
	return InitGreet(p, cfg)
}

That package init() registers three things with the framework the moment your command is imported, with no central setup file for you to edit: the initialiser itself, an init greet subcommand so the user can reconfigure just this command later, and a --skip-greet flag on the main init. IsConfigured is how the framework avoids nagging: if the greet key is already in the config, init leaves it be and moves on.

All of that is generated for you. The one piece that’s yours is the InitGreet function in main.go, which starts as a stub:

func InitGreet(p *props.Props, cfg config.Containable) error {
	// TODO: Implement custom initialization logic for greet
	return nil
}

Fill it in with whatever the setup needs. go-tool-base leans on huh for prompts, the same library its own GitHub and AI setup use, so a one-question form looks like this:

func InitGreet(p *props.Props, cfg config.Containable) error {
	var greeting string

	form := huh.NewForm(
		huh.NewGroup(
			huh.NewInput().
				Title("What greeting should greet use?").
				Value(&greeting),
		),
	)
	if err := form.Run(); err != nil {
		return err
	}

	cfg.Set("greet.greeting", greeting)
	return nil
}

Set the value on cfg and you’re done. After the initialisers run, init writes the whole config out to disk, so the answer persists into ~/.mytool/config.yaml with everything else. Run mytool init on a fresh machine now and it stops to ask for the greeting; run it again and it sails past, because IsConfigured sees the key is already there. Need to redo just this one command’s setup? mytool init greet. The framework hands each command its own setup step, its own subcommand and its own skip flag, and asks you for a single function in return. That’s the trade worth making: static defaults in your embedded YAML, anything that needs a human in an initialiser.

Overriding: the environment and layered files

With a config file in place, the other sources come into their own. The quickest override is an environment variable. Remember the prefix you set when scaffolding in part 1: hello.greeting maps to MYTOOL_HELLO_GREETING, the prefix and key joined up, uppercased, dots turned to underscores:

MYTOOL_HELLO_GREETING="Hello from mytool" ./bin/mytool hello

INFO Hello from mytool

You didn’t register that variable anywhere; the config system binds it for you. The prefix is what keeps it from colliding with some other tool’s LOG_LEVEL on the same machine, which is exactly why it’s worth having.

Files are the other half, and they’re where that precedence list earns a closer look. A single config file is fine until two people, or two machines, want slightly different settings, and then you’re copying files around by hand. The --config flag fixes that: pass it more than once and the tool merges the files in order.

./bin/mytool hello \
 --config ./config.yaml \
 --config ./config.local.yaml

Between the files you name, the rule is later wins on a clash, and every key that doesn’t clash is kept. If config.yaml sets hello.greeting: Hello and config.local.yaml sets hello.greeting: Oi, you get Oi, but keys that appear in only one file survive untouched. It’s a merge between them, not a replacement.

The edge to remember is what --config does to the default locations: it replaces them. The moment you name a file, ~/.mytool/config.yaml drops out of the picture unless you name it too. So you pass the whole stack you want, a shared base and a local override together, and let precedence settle it. Commit a config.yaml with the team’s settings, keep an untracked config.local.yaml for your own, run with both, and your local tweaks win without anyone editing a shared file. Leave --config off and you’re back on the defaults init wrote: ~/.mytool/config.yaml plus that machine-wide /etc/mytool/config.yaml if it’s there. Whichever set of files you land on, environment variables and flags still sit on top.

The typo that does nothing

Now for the failure I keep circling. Say you want to change the greeting. Open your config, but fat-finger the key:

hello:
 greting: Oi  # meant to be greeting

Run it, and you get a blank line. The greeting you set never applied: the misspelled key was read, matched nothing, and was silently dropped, and the real greeting is now nowhere to be found. Nothing said a word. For a greeting it’s a shrug. For a timeout or a retry count it’s the bug you chase at 2am, and I wrote up the why of it in the config key that quietly did nothing.

go-tool-base won’t catch this for you by default, and that’s a choice rather than an oversight. There’s no central schema that knows every key your tool could ever take, because keys belong to the commands that use them. What you get instead is a way to opt a command in, so it validates its own slice and nobody else’s.

Making mistakes loud

Tell the generator you want validation for a command and it scaffolds exactly this (gtb generate command --name hello --with-config-validation). Since hello already exists, it’s a small file to add by hand. Create pkg/cmd/hello/config.go:

package hello

import "gitlab.com/phpboyscout/go-tool-base/pkg/config"

// HelloConfig describes the config keys the hello command consumes.
type HelloConfig struct {
	Greeting string `config:"hello.greeting" validate:"required"`
	Style string `config:"hello.style" enum:"plain,loud" default:"plain"`
}

// ValidateHelloConfig checks the hello config against its schema.
func ValidateHelloConfig(cfg config.Containable) error {
	return config.ValidateStruct[HelloConfig](cfg)
}

The tags carry the rules. validate:"required" means the key has to be present and non-empty. enum:"plain,loud" means style has to be one of those two words. config.ValidateStruct[HelloConfig] does the rest: it derives a schema from those tags and checks the config against it, returning a readable error if anything is off. It takes props.Config as it is, the Containable interface, so there’s no casting to a concrete type. Call it at the top of RunHello, before you trust any of the values, and use the style while you’re there:

func RunHello(ctx context.Context, props *props.Props, opts *HelloOptions, args []string) error {
	if err := ValidateHelloConfig(props.Config); err != nil {
		return err
	}

	greeting := props.Config.GetString("hello.greeting")
	if props.Config.GetString("hello.style") == "loud" {
		greeting = strings.ToUpper(greeting)
	}

	props.Logger.Info(greeting)
	return nil
}

(You’ll add strings to the imports at the top of main.go.)

Now make a real mistake. Set the style to something that isn’t allowed:

hello:
 greeting: Hello
 style: shout

ERRO config validation failed:
 hello.style: value "shout" is not allowed (hint: allowed values: plain, loud)

That’s the difference. The command stops and tells you the key, the bad value, and what it would have accepted. The same check catches a misspelled greeting: the moment the real key goes missing, required fails with hello.greeting: required field is missing instead of quietly running on nothing. Set style: loud and you get HELLO, because the value finally passes and the code downstream can trust it.

If you switch on the optional config feature (it isn’t in the default set, so you opt into it), you also get a ready-made mytool config validate command that runs these checks without you wiring anything into a command at all. Either way, the principle holds: the program already knows what good config looks like, so make it say so when the config is bad.

The upshot

Your hello command now reads a real setting, ships a sensible default that init writes into place, honours overrides from the environment and from layered files in a predictable order, and refuses to run on a value it doesn’t understand. That’s most of what configuration ever needs to be, and you wrote almost none of the machinery.

One thing I’ve skipped: config can also reload while the tool is running, so a long-lived process picks up a changed file without a restart. That’s its own capability with its own moving parts, and I pulled it apart in reloading config without a restart if you need it.

Next part, we give the tool something to do with all this config: we turn it into an AI tool, with a chat command and an MCP server. Until then, go add a couple of validated settings to your own commands. You’ve got the shape of it now.

Building a CLI with go-tool-base, part 1: scaffold and your first command

Wed, 20 May 2026 00:00:00 +0000

Every time I start a new Go CLI, the first hour goes the same way, and none of it is the actual tool. Config loading. A logger. An update command. An error path that prints something a human can act on. A help system. I built go-tool-base so I’d never write that hour again, and I’ve spent a good few posts explaining how the pieces work inside. This series is the other half: how you use it. By the end you’ll have a real CLI with all that wiring for free. This part scaffolds one and gives it its first command.

One note on shape before we start: each part stands on its own. Finish this one and you’ve got a working, buildable tool. Later parts add configuration, AI, self-update and telemetry, one at a time. Where you want to know how a piece works underneath, I’ll link the deep-dive as we go.

Install the gtb CLI

go-tool-base ships an automation CLI called gtb. Install it with the script from the installation docs:

curl -sSL \
 "https://gitlab.com/phpboyscout/go-tool-base/-/raw/main/install.sh" | bash

That fetches a pre-built release, embedded docs and all, and drops gtb in ~/.local/bin, so make sure that’s on your $PATH. Then check it’s there:

gtb version

One thing to get out of the way before we build: versions. As I write this, gtb version prints go-tool-base v0.6.0, and that’s what every command and snippet in this series is verified against. It’s a young tool that’s still moving quickly, and the install script always pulls the latest release, so if you’re reading this later and something doesn’t line up, a newer version is the likeliest reason. When a release changes something that matters to this series, I’ll cover it in a follow-up.

Scaffold a project

One command stands up a whole project, and gtb gives you two ways to drive it.

The direct way, with flags, is good for scripting and for repeating a setup exactly:

gtb generate project \
 --name mytool \
 --repo myorg/mytool \
 --description "My CLI tool" \
 --env-prefix MYTOOL \
 --path ./mytool

--env-prefix is worth setting now: it’s the prefix for the environment variables that can override your config later (so MYTOOL_LOG_LEVEL rather than a bare LOG_LEVEL that would clash with every other tool on the box). The wizard defaults it to your tool’s name in capitals; with flags it’s worth being explicit. We’ll lean on it in part 2.

(gtb generate cli is the same command, if you prefer that name.) Or leave the flags off and gtb walks you through an interactive prompt instead, which is the gentler way the first time:

gtb generate project

Either way, one of the choices is worth calling out now, because it explains something you’ll see in a minute: features. go-tool-base bundles a set of ready-made commands, self-update, embedded docs, a doctor health check, an MCP server, a changelog, OS-keychain storage, and you choose which ones your tool ships with at generation time, either through the --features flag or a checklist in the wizard. The default set is a sensible starting point, and you can add or drop features later. That is why, a moment from now, your brand-new tool already answers --help with commands you never wrote. The full flag list is in the generate reference.

What you just got

cd mytool and look around. It’s a complete, releasable project, not a hello-world:

mytool/
├── cmd/mytool/main.go # entry point
├── pkg/cmd/root/
│ ├── cmd.go # builds Props, wires the root command
│ └── assets/init/config.yaml # embedded default config
├── internal/version/version.go # version info, stamped at release
├── .gtb/manifest.yaml # the generator's record of your command tree
├── .github/workflows/ # lint, test, docs, release pipelines
├── justfile # build / test / lint / docs tasks
├── go.mod # with `go tool` deps pinned
└── ... # .golangci.yaml, .goreleaser.yaml, README, CHANGELOG

One file there is worth understanding before anything else: .gtb/manifest.yaml. It is the generator’s source of truth, a record of every command your tool has, how they nest, and a content hash of each generated file. You won’t edit it by hand, but gtb reads and rewrites it constantly. It is how the generator knows what your command tree looks like, and how it can tell whether you have changed a file it owns. Think of it as the map the generator builds from: it’s committed to git for you, and as long as it’s there, your tool’s structure stays reproducible. We’ll see it earn its keep when we regenerate.

The entry point, by contrast, is tiny, because the framework does the lifting. Here’s the generated cmd/mytool/main.go in full:

// Code generated by gtb. DO NOT EDIT.

package main

import (
	"mytool/internal/version"

	gtbRoot "gitlab.com/phpboyscout/go-tool-base/pkg/cmd/root"
	"mytool/pkg/cmd/root"
)

func main() {
	rootCmd, p := root.NewCmdRoot(version.Get())
	gtbRoot.Execute(rootCmd, p)
}

Two lines of body. root.NewCmdRoot (in your pkg/cmd/root/cmd.go) builds a Props, the container that carries the logger, config, filesystem and version to every command. gtbRoot.Execute runs it and routes any failure through one consistent error handler, so there’s no os.Exit scattered about. Note the DO NOT EDIT header: main.go and the root cmd.go belong to the generator. Your code goes elsewhere, which matters in a minute.

Build it and you already have a working CLI:

just build # or: go build -o bin/mytool ./cmd/mytool
./bin/mytool --help

You’ll see the built-in commands from the features you picked, update, docs, doctor and the rest, with not a line written by you.

There’s one step before those commands will actually run. Try one, say ./bin/mytool docs, and the tool stops with please run init: it has no configuration yet and won’t guess at one. So give it some:

./bin/mytool init

That writes ~/.mytool/config.yaml from the defaults your tool ships with, and now its commands run. (init is itself one of the features. You can switch it off for a tool that should run straight from its built-in defaults with no file at all, but leave it on for now.) Part 2 takes configuration apart properly; for now, init once and carry on.

Add your first command

Don’t hand-roll a command file. gtb generates the boilerplate and leaves you the logic:

gtb generate command --name hello --short "Say hello"

That creates two files (see the command reference):

pkg/cmd/hello/cmd.go (generated, DO NOT EDIT): the options struct, flag wiring, and the NewCmdHello(props *props.Props) constructor.
pkg/cmd/hello/main.go (yours): a RunHello function, where all your real business logic goes.

The split is the whole point. Open pkg/cmd/hello/main.go and write what the command does:

func RunHello(ctx context.Context, props *props.Props, opts *HelloOptions, args []string) error {
	props.Logger.Info("hello from mytool")
	return nil
}

Rebuild, and the command is wired into the tree:

just build
./bin/mytool hello

You never touched the root command to register it. gtb recorded hello in that .gtb/manifest.yaml and wired it in for you. (If you’d rather wire commands by hand against the library directly, the custom-commands how-to shows that path; the generated route is the one this series follows.)

Regenerate without losing your work

Here’s the bit people are right to be wary of. If the generator owns cmd.go and the root wiring, what happens when it runs again, after you’ve made changes? And it runs often: every gtb generate command rebuilds the wiring.

gtb regenerate project

Your edits survive, and not by luck. Three separate things protect them:

Your logic sits in a file the generator never rewrites. Command logic lives in main.go; only the boilerplate cmd.go is regenerated. The split isn’t cosmetic, it’s the contract.
It notices if you edited a generated file. That manifest stores a content hash of every generated file, so if you’ve changed one, regeneration stops and asks before overwriting rather than silently stamping over you.
You can fence files off entirely. A gitignore-style .gtb/ignore tells the generator to leave specific paths alone, even under --force.

I wrote up how that edit-preserving diff actually works if you want the mechanism; the regenerate reference has the flags. For now, the thing to trust: scaffolding here is not a one-way door. You keep regenerating as the tool grows, and your edits stay put.

Where this leaves you

A few minutes in, you have a real CLI: config, logging, a consistent error path, self-update, embedded docs and a release pipeline, none of it written by you, plus your own hello command and the confidence to regenerate without fear. That’s the head start go-tool-base exists to give.

Next part: configuration. Typed settings, defaults the tool ships with, and how to turn a misspelled config key from a silent shrug into an error that tells you what you got wrong rather than a mystery you debug at 2am. Until then, go add a few more commands. You’ve got the pattern now.

Reloading config without a restart

Mon, 27 Apr 2026 00:00:00 +0000

A config file changes. Someone edits a setting, rotates a credential, flips a feature flag. How does the running process find out? For most processes the answer is blunt: it doesn’t, until you restart it. For a short-lived CLI that’s completely fine. For a long-running service, “just restart it” is a much bigger ask than it sounds.

The default answer is a restart

Configuration lives in a file. The file changes: someone edits a setting, rotates a credential, flips a feature flag. How does the running process find out?

Overwhelmingly, the answer is that it doesn’t. A process reads its config once, at startup, and that snapshot is frozen for the life of the process. Change the file and nothing happens until you restart, at which point a fresh process reads the fresh file.

For a short-lived CLI invocation that’s completely fine. It reads config, does its job, exits, and the next invocation reads whatever the file says then. But the same frameworks are also used to build long-running services, and for a service “just restart it” is not the small thing it sounds like.

What a restart actually costs

Restarting a long-running service means every open connection drops. Any in-flight request is lost, or has to be retried by whoever sent it. Caches that took real time to warm are cold again. There’s a window, short but real, where the service simply isn’t serving.

If the thing you changed was a log level, or a feature flag, or a timeout, you’ve paid a disruption wildly out of proportion to the change. And the calculation only gets worse as the service gets more important, because the services you least want to bounce on a whim are exactly the ones that matter most.

Hot-reload: re-read in place

Hot-reload is the alternative, and both go-tool-base and rust-tool-base support it.

The process doesn’t read config once and freeze it. It watches the config file. When the file changes, it re-reads it, re-applies it, and carries on running. No new process, no dropped connections, no cold start. The change lands in the live process.

The shape is the same in both frameworks:

A file watcher notices the config file changed. Underneath, this is the operating system’s own file-notification facility, inotify on Linux and its equivalents elsewhere. rust-tool-base reaches it through the notify crate; go-tool-base, through the watcher built into Viper.
A debounce step waits for the writes to settle. Saving a file is often several separate operations, and you don’t want to reload three times for one edit.
The config is re-parsed from disk.
The new config is swapped in atomically.
Observers are notified, so the subsystems that care can react.

Steps four and five are the ones worth slowing down on, because they’re where a naive hot-reload quietly goes wrong.

The two details that make it safe

The atomic swap. You do not mutate the live config object in place. A reader on another thread, partway through reading it, would see a torn mix of old and new values, and that’s a genuinely nasty class of bug. Instead the process builds a new, complete config value and swaps the pointer to it in a single atomic operation. Any reader sees either the entire old config or the entire new one, never a blend. rust-tool-base does this with arc-swap; go-tool-base does the equivalent. Reads stay cheap and lock-free, and an update is one pointer swap.

The observer notification. Re-reading the file isn’t the end of the job. Some subsystems have to do something when config changes: a connection pool resizes, a logger changes level, a rate limiter takes a new ceiling. So a hot-reload system has to let those subsystems subscribe. rust-tool-base hands observers a watch::Receiver, a channel that always holds the latest value; go-tool-base exposes an Observable interface. A subsystem subscribes once and reacts every time config changes, for the life of the process.

Where this earns its keep: a Kubernetes pod

Hot-reload is a nicety on a developer’s laptop. Inside a Kubernetes pod it becomes genuinely valuable, and the reason is a neat fit between how Kubernetes delivers config and how a file watcher works.

In Kubernetes you don’t usually bake configuration into the container image. It lives in ConfigMap and Secret objects, and the clean way to consume them is to mount them as volumes. Mount a ConfigMap as a volume and each key becomes a file in the pod’s filesystem.

Here’s the part that connects to everything above. When you update that ConfigMap or Secret, Kubernetes does not restart your pod. The kubelet notices the object changed and rewrites the projected files inside the still-running pod. The files on disk change underneath a process that never stopped.

That file rewrite is exactly the event a hot-reload watcher exists to catch. So the whole chain becomes:

You kubectl apply an updated ConfigMap, or rotate a Secret.
The kubelet updates the projected files inside the pod.
The framework’s file watcher sees the write.
The config is re-parsed, swapped in atomically, and observers are notified.
The new configuration is live, and the pod never cycled.

You’ve changed a running service, in a running pod, with no rollout, nothing terminated and recreated, no dropped traffic. Rotate a database credential, raise a log level to debug an incident in progress, flip a feature flag: all of it live. For a service where a restart is the very thing you’re trying hard to avoid, the kind of long-running service these frameworks are built for, that’s the difference between a config change being routine and being an event.

The caveats

Two things, so this doesn’t read as magic.

First, not everything can be hot-reloaded. Some configuration genuinely needs a restart: the port a server binds to, the size of a thread pool, anything wired up exactly once at process start. Hot-reload covers the large category of settings a subsystem can re-read and re-apply; it doesn’t abolish restarts. A config system worth its salt is clear about which settings are live and which are not.

Second, a Kubernetes gotcha that catches people out. The in-place file update happens for ConfigMaps and Secrets mounted as volumes. Consume the same ConfigMap as environment variables instead, and those are fixed when the container starts and never update, short of a restart. If you want hot-reload in a pod, mount config and secrets as files, not env vars. And even with volumes the update isn’t instant: the kubelet syncs on a period, around a minute by default, so a reload is “within a minute or so”, not “the moment you hit apply”.

What it comes down to

A config file changes, and the default way to pick it up is to restart the process. For a long-running service that restart costs dropped connections, lost work and a cold start, often for a change as small as a log level.

go-tool-base and rust-tool-base both support hot-reload instead: a file watcher catches the change, the config is re-parsed and swapped in atomically so no reader sees torn state, and observers are notified so subsystems can react, all in a live process. The setting where it pays off most is a Kubernetes pod, where ConfigMaps and Secrets mounted as volumes are rewritten in place by the kubelet and the watcher catches that write directly. Mount them as volumes rather than env vars, allow for the kubelet’s sync delay, accept that some settings still need a restart, and within those limits “the config changed” stops meaning “cycle the pod”.

Verifying your own downloads: how I solved it for self-updating CLI tools

Fri, 24 Apr 2026 00:00:00 +0000

Way back in the introduction I promised I’d come back to the self-update integrity checks. Here we are. And the starting point is a slightly uncomfortable admission: for a good long while, go-tool-base’s update command was the most trusting line of code in the entire tool.

The most trusting line of code in the tool

Self-update is a lovely feature. The user runs yourtool update, the tool fetches the latest release, swaps itself out, and they’re current. go-tool-base has had this since early on, wired to GitHub, GitLab, Bitbucket, Gitea and a few others.

But look closely at what that feature actually does. It reaches out to the internet, pulls down a file, and then replaces the executable that’s currently running with that file. The next time the user invokes the tool, they’re running whatever those bytes turned out to be.

The original implementation downloaded the release asset over HTTPS and extracted it. HTTPS gets you transport security: the bytes weren’t tampered with in flight. It tells you nothing about whether the bytes were right when they left, or whether they’re even the bytes you meant to fetch. A truncated download, a CDN cache serving a mangled object, a release asset that got swapped after the fact… HTTPS waves all of those straight through. For the one operation in the whole tool that replaces the binary, “we didn’t check” is an uncomfortable place to be sitting.

GoReleaser already does half the job

The good news is that the build side was already producing exactly what I needed. GoReleaser, which builds go-tool-base’s releases, generates a checksums.txt for every release: one SHA-256 per published artefact, the same format sha256sum emits. It was sitting right there as a release asset and nothing was reading it.

So Phase 1 of the integrity work is exactly that: read it.

When update downloads the platform binary, it now also fetches checksums.txt from the same release, looks up the entry for the asset it just pulled, and compares the SHA-256 of the downloaded bytes against the expected hash before anything gets extracted or installed. Mismatch, and the update aborts before it has so much as touched the installed binary. The hash comparison runs in constant time, which is more defence-in-depth than strictly necessary here, but it costs nothing and means every hash comparison in the codebase is the same and reassuringly audit-boring.

Fail open, or fail closed?

The interesting design question wasn’t the hashing. It was: what do you do when there is no checksums.txt?

Plenty of older releases predate this feature. A release might have been cut by hand without GoReleaser. If go-tool-base flatly refused to update whenever a manifest was missing, the very act of shipping this feature would brick the update path for every existing tool the moment they upgraded into it. That’s a cure worse than the disease.

So the default is fail-open: no manifest, log a clear warning, proceed. It matches how the existing offline-update path already behaved with its optional .sha256 sidecar, and it keeps upgrades working.

Fail-open as a default is not the same as fail-open being right for everyone, though. A security-sensitive tool should be able to say “no manifest, no update, full stop”. Two ways to get there:

Tool authors flip a compile-time switch (setup.DefaultRequireChecksum = true in main()) and their binary ships fail-closed from day one.
End users override either way through config (update.require_checksum) or an environment variable.

go-tool-base itself ships with the strict setting turned on, because a tool whose entire job is being a careful framework should hold itself to the stricter bar.

The caveat

Security features oversell themselves constantly, so here is the limit, stated plainly.

A checksum hosted next to the binary it describes protects you from accidents. Corruption, truncation, a CDN serving stale junk, a release asset that got partially clobbered. It does not protect you from a determined attacker who’s compromised the release platform itself. If someone can replace the binary, they can replace checksums.txt in the same breath, and your tool will cheerfully verify a malicious download against a malicious manifest and pronounce it good.

That’s not a flaw in the implementation. It’s the inherent ceiling of same-origin integrity: the manifest and the artefact share a trust root, so they fall together. Closing that gap needs a signature whose trust root is somewhere the release platform can’t reach, a key the attacker doesn’t have. That’s the next phase of this work, and it’s a bigger piece: GPG-signing the manifest, with the public half both embedded in the binary and published independently so a single platform compromise isn’t enough.

Phase 1 is the floor, not the ceiling. But it’s a floor worth having, because the overwhelming majority of real-world “the download was wrong” incidents are accidents, not attacks, and accidents are exactly what a same-origin checksum catches.

Pulling it together

The update command is the most trusting code in a self-updating tool: it fetches bytes from the internet and then becomes them. go-tool-base now verifies the SHA-256 of every self-update download against the release’s own checksums.txt before installing. It fails open by default so shipping the feature doesn’t strand anyone on an un-updatable version, fails closed for tool authors who ask (go-tool-base itself does), and stays honest that a same-origin checksum stops accidents, not a platform compromise.

Verifying your own downloads is a low bar. The point is that the previous height of that bar was zero.

What survives a port, and what doesn't

Thu, 23 Apr 2026 00:00:00 +0000

Rebuilding go-tool-base in Rust turned out to be the most honest design review I’ve ever sat through, and I didn’t have to do anything except keep going. Porting a framework into a language with completely different idioms forces a separation you can’t fake: the parts that survive the move are design, and the parts that don’t are just habit.

Two columns

When you port a system between languages that don’t share idioms, every piece of it sorts itself into one of two columns, without you having to make the call.

In the first column is the outcome a piece of the design produces: every command receives the framework’s services, configuration is layered with a fixed precedence, commands register themselves, errors carry guidance to the user. In the second column is the mechanism that produced that outcome in the original language.

Things in the first column survive the port. You rebuild them, differently, because the tool genuinely needs them. Things in the second column do not survive. You find their replacement, and the Go version turns out to have been one valid implementation of an idea, not the idea itself. Doing this for go-tool-base, mechanism by mechanism, was more honest about my own design than any amount of sitting and staring at it would have been.

The container

go-tool-base hands every command a Props struct. It carries the logger, the config, the assets, the filesystem handle. Some of it is reached through loosely-typed accessors. It works well, and I wrote a whole post about it.

The outcome is column one: a command should receive one object, and that object should carry the framework’s services so the command doesn’t go assembling them itself. That survived. RTB hands every command an App.

The loosely-typed accessors were column two. In Rust an App is a plain struct with concrete fields, each one an Arc<T> so a clone is a few atomic increments rather than a deep copy. Nothing is keyed by string. Nothing is fetched by name and asserted to a type. The thing the container is for survived; the way Go expressed it did not.

Registration

A go-tool-base command self-registers using a package-level init() function, which Go runs before main() and which appends the command to a global slice.

The outcome, column one, is that a command lives in its own file and inserts itself into the framework with no central list to edit. That’s genuinely worth keeping.

The init() mechanism is column two, and Rust doesn’t even offer it: Rust deliberately has no code that runs before main(). The replacement is link-time registration through distributed slices, which gets its own post next. Same outcome, no global mutable state, assembled by the linker rather than by a startup phase.

Configuration

go-tool-base layers configuration with a precedence: flags over environment over file over defaults. Some of it is read back through key lookups.

The layering and the precedence are column one. They survived exactly. RTB layers config with the same ordering.

The key lookups were column two. In Rust the merged configuration is deserialised into your own serde struct, so a config value is a typed field you access like any other field, and a typo is a compile error instead of a missing key at runtime. The precedence survived; reading values back out of a string-keyed bag did not.

The error path

go-tool-base routes every error through one handler so presentation is consistent, which I also wrote up.

One consistent exit for errors is column one. It survived. What didn’t survive was the handler: RTB has no error-handler object at all, because Rust’s own return-from-main convention plus a report hook does the job the handler was built to do. That one has its own post too.

What the exercise was actually worth

Every mechanism told the same story. The container, the registration, the config access, the error path, the cancellation signal that go-tool-base carries on a context.Context and RTB carries on a CancellationToken. In every case the thing it achieved walked across to Rust untouched, and the Go code that achieved it was left behind.

That’s the useful result. Before this port I couldn’t have told you, for any given pattern in go-tool-base, whether it was load-bearing design or just the idiomatic Go way to write it that day. Now I can, because each one was forced to prove itself by being rebuilt from nothing in a language that flatly wouldn’t accept the original. Whatever survived was real. Whatever I had to replace was always replaceable, which means it was never really the point.

The upshot

Porting a framework into a language with different idioms separates design from habit for free. The outcome a pattern produces is design, and it survives the move. The mechanism that produced it is idiom, and it gets left behind for the new language’s equivalent.

go-tool-base’s Props bag, its init() registration, its key-based config access and its error handler were all idiom. The single context object, self-registration, layered precedence and a consistent error exit were all design, and all four came through to RTB intact. The next three posts take the most interesting replacements one at a time, starting with how a Rust command registers itself when the language won’t run anything before main.

rust-tool-base: the same idea, in a language that argues back

Wed, 22 Apr 2026 00:00:00 +0000

I built go-tool-base because I was sick of rebuilding the same CLI scaffolding every time I started a new Go tool. You’d think that would have taught me a lesson about doing things more than once. Apparently not, because I’ve now started building rust-tool-base: the same idea, the same itch, for Rust.

In my defence, there’s method in it.

The same itch, a different language

go-tool-base exists because I kept writing the same couple of hundred lines of wiring every time I started a new Go CLI. Config loading, logging setup, an update check, an error path, a help system. None of it was the tool. All of it had to be there before the tool could be.

Lately I’ve been learning Rust, and two things collided. The first is how I tend to learn a language. I’ve always picked them up reasonably quickly, and the way I do it isn’t with a tutorial that builds a toy, it’s by rebuilding something whose shape I already know cold, so that every decision is about the language rather than the problem. The second is that every time I started a Rust CLI of any size, I hit the very same gap I’d already filled once in Go.

So rather than learn Rust on a throwaway, I decided to learn it by building rust-tool-base: the same idea, the same niche, for Rust.

One housekeeping note before the series gets going. You don’t need to write Rust to follow it. The posts lean on a handful of language ideas, and rather than stop and re-explain each one mid-flow, I’ve gathered them into a short primer. If a bit of syntax trips you up along the way, that’s where to look. If you already write Rust, ignore me and read on.

The gap in Rust

The Rust ecosystem has a well-earned reputation for sharp, focused crates and a deliberate shortage of big opinionated frameworks. clap for argument parsing, figment for layered config, tracing for logging, miette for errors, ratatui for terminal UI, reqwest and tokio underneath. Each of them is genuinely best-in-class.

What nobody hands you is the assembly. Wiring those into one coherent product, and then adding self-update, AI integration, an MCP server, embedded documentation, credential handling, telemetry and a scaffolder, is real work, and it’s the same work on every project.

The closest existing neighbours stop short of it. cli-batteries is a thin preamble: argument parsing plus a logging subscriber plus panic and signal handling. starbase has a proper session and lifecycle model but is CLI-agnostic and shaped around the moonrepo tooling it came from. cargo-dist and cargo-release are about release packaging, not the runtime. Good tools, all of them, but none is the opinionated, full-lifecycle, scaffolded base that go-tool-base is in the Go world. That space is empty, and rust-tool-base is built to fill it.

Why it is not a port

The obvious way to build this would be to open go-tool-base and translate it file by file. I’m not doing that, and the reason matters enough that it’s the rule the whole project is built around.

go-tool-base is full of Go. It leans on a Props struct that carries the framework’s services in loosely-typed fields. It configures things with functional options. It registers commands using package-level init(). It threads a context.Context through every call. Those are all good, idiomatic Go. Transliterated into Rust they’d become code that argues with the compiler on every single line, because Rust has its own answers to every one of those problems and they are emphatically not the Go answers.

So rust-tool-base reaches the same outcomes by Rust’s means. Commands still self-register, but through link-time machinery instead of init(). There’s still one context object per command, but it’s strongly typed rather than a loosely-keyed bag. Configuration is still layered, but it lands in your own typed struct instead of a string-keyed lookup. Same philosophy, same shape of product, an entirely different ecosystem underneath. The README says it plainly: it’s a sibling, not a port.

Why do it twice at all

Three reasons, and they reinforce each other.

The first is plain usefulness. The next time I want a Rust CLI tool, I want the same head start go-tool-base already gives me in Go.

The second is the learning. Rebuilding a system I understand forces me to meet Rust’s idioms where they actually bite, not where a tutorial gently stages them. You learn ownership properly when a real design is pushing back at you.

The third is the one I didn’t expect, and it’s the subject of the next post. Building the same framework twice, in two languages, turns out to be the cleanest way to find out which of your original decisions were genuine design and which were merely idiom. The design survives the move. The idiom does not. Sorting one from the other has been the most interesting part so far.

Boiling it down

rust-tool-base is the Rust sibling of go-tool-base: the same batteries-included, scaffolded, opinionated CLI framework, aimed at the same gap, which in Rust is the gap between a pile of excellent crates and a coherent product.

It’s not a port. Transliterating Go idioms into Rust produces code that fights the language, so RTB reaches the same outcomes through Rust’s own mechanisms instead. The posts after this one walk through the specific cases: how commands register, how the builder works, how errors are reported, and a few things RTB can do that the Go version structurally can’t. First, though, the thing the exercise taught me about my own design.

The blank import that keeps a dependency out of your binary

Wed, 22 Apr 2026 00:00:00 +0000

go-tool-base can stash your credentials in the OS keychain, which most people building on it are perfectly happy about. But some of them ship into regulated and air-gapped environments where the binary isn’t permitted to contain keychain or session-bus code at all… not dormant, not unused, simply not there.

So I had a feature most users want and a minority must be able to provably not have. The way I ended up solving it is one of my favourite little bits of honest Go.

A feature some users have to be able to not have

go-tool-base needs somewhere to keep secrets: AI provider keys, VCS tokens, the occasional app password. The best home for those on a developer’s machine is the operating system’s own keychain. macOS Keychain, GNOME Keyring or KWallet on Linux via the Secret Service, Windows Credential Manager. So I wanted go-tool-base to support all three. (This is the keychain mode I mentioned back in the credentials post, finally getting the explanation I promised it.)

The Go library for that is go-keyring, and it’s good. The catch is what it drags in behind it. On Linux it talks to the Secret Service over D-Bus, which means godbus. On Windows it pulls wincred. Perfectly reasonable dependencies for a desktop tool.

Now here’s the constraint that made this interesting. Some of the people building tools on go-tool-base don’t ship to developer laptops. They ship into regulated sectors and air-gapped deployments where a security review will scan the binary, enumerate every dependency, and ask pointed questions about anything that does inter-process communication. For those builds, “the keychain code is there but we never call it” is not an acceptable answer. The reviewer’s position, and it’s a fair one, is that code which isn’t in the binary cannot be a finding.

So I had a feature that most users want, and a minority of users must be able to provably not have. Same framework, same release.

Why I didn’t reach for a build tag

The obvious Go answer is a build tag. Compile with -tags keychain to get it, leave the tag off to not. I started down that road. I even spent a while on an inverted version, a nokeychain tag, on the theory that the regulated build should be the one that has to ask, so a forgotten flag fails safe.

It works. It also isn’t very nice. Build tags are invisible at the call site. Nothing in the source tells you that a file only exists in some builds. The two worlds drift, because the tagged-out path isn’t compiled in your normal editor session and quietly rots. And the ergonomics for a downstream consumer are poor: every tool built on go-tool-base would have to know the right magic incantation and thread it through their own release pipeline correctly, forever.

I tried a second approach too: pull the keychain backend out into a completely separate Go module. That genuinely solves the dependency question (a module you don’t require can’t contribute to your go.sum). But a separate module for one backend is clunky. Separate versioning, separate release, separate repo, all for a single file’s worth of behaviour. It felt like using a shipping container to post a letter.

The shape that actually fits: a registry and an `init()`

The version I’m happy with leans on two boring, well-worn Go mechanisms and lets them do something quietly clever together.

First, pkg/credentials defines a Backend interface and a registry. By default the registry holds a stub backend that politely returns “unsupported” for everything. The framework only ever talks to the registered backend, whatever that happens to be.

Second, the keychain implementation lives in its own package, pkg/credentials/keychain, still inside the same module, no separate release to manage. That package has an init() that registers its go-keyring-backed backend:

//nolint:gochecknoinits // registration via import is the whole point
func init() {
 credentials.RegisterBackend(Backend{})
}

And go-keyring, godbus, wincred, the whole IPC dependency chain, are only imported by that package.

Now the trick. To switch keychain support on, you import the package. You don’t have to use anything from it. A blank import is enough, because a blank import still runs the package’s init():

// cmd/gtb/keychain.go - the entire file.
package main

import _ "gitlab.com/phpboyscout/go-tool-base/pkg/credentials/keychain"

That single line is the on/off switch for the shipped gtb binary. The blank import means init() runs, the keychain backend registers itself, and credential operations start routing through the OS keychain. No flag, no tag, no config.

The part that makes it provable

Here’s why this beats the build tag, and it comes down to one guarantee in the Go toolchain: the linker only includes packages that are actually imported.

If cmd/gtb/keychain.go exists, the keychain package is in the import graph, so go-keyring, godbus and wincred are linked in. Delete that one file and rebuild, and the keychain package is no longer reachable from main. The linker performs dead-code elimination, and the entire go-keyring chain is gone. Not dormant. Not present-but-unused. Absent from the binary.

That’s the bit a regulated build needs. It isn’t a promise that the code won’t run. It’s a structural fact that the code isn’t there, and you can hand a security reviewer an SBOM that proves it. go-keyring won’t appear, because it genuinely isn’t linked.

For a downstream tool built on go-tool-base the story is the same, and just as cheap. Want keychain support? Add the one-line blank import to your own cmd package. Must ship keychain-free? Don’t. Your binary’s dependency graph follows your import graph, exactly as Go always promised it would. The default (no import) is the locked-down one, which is the right way round for a safety property.

Why I like this more than I expected to

Build tags hide a decision in the compiler invocation. This pattern puts the decision in the source, as an import, where it’s greppable, obvious in code review, and impossible to get subtly wrong. There’s a real file called keychain.go whose entire content is one import, and it reads as exactly what it is: a switch.

It’s also just honest Go. No reflection, no plugin loader, no clever runtime. A registry, an init(), and the linker doing the one job it’s always done. The cleverness, such as it is, is in the arrangement, not in any individual piece.

Stepping back

go-tool-base needed OS keychain support for the many, and a way to provably exclude it for the few. Build tags could express the toggle but hid it in the build invocation and rotted in the dark. A separate module solved the dependency question but was far too much machinery for one backend.

Putting the keychain backend in its own package, activated by a blank import _ that fires its init(), gets you both: a one-line, in-source, code-reviewable switch, and, because the linker only links what’s imported, a build with the import omitted that contains none of the keychain dependency chain. Provable absence, not promised disuse.

If you’re carrying an optional dependency that some of your users need gone rather than merely idle, this is the pattern. Let the import graph be the feature flag.

Where should a CLI keep your API keys?

Mon, 20 Apr 2026 00:00:00 +0000

Your CLI tool needs the user’s API key. It has to come from somewhere, and it has to survive between runs, so the obvious move is to ask once and write it into the config file. One tidy api_key: line. Job done.

It works beautifully on the first afternoon. And then, months later, it’s quietly become a liability nobody actually decided to create.

The config file that quietly becomes a liability

Your CLI tool needs the user’s API key. It has to come from somewhere, and it has to survive between invocations, so the obvious move is to ask once and write it into the tool’s config file. ~/.config/yourtool/config.yaml, a nice api_key: line, done.

It works on the first afternoon. It keeps working. And then, slowly, it becomes a problem nobody decided to create.

The config file gets committed to a dotfiles repo. It gets caught in a tar of someone’s home directory that lands in a backup bucket. It scrolls past in a screen share. It sits, world-readable, on a shared build box. None of these are exotic. They’re just a Tuesday. The plaintext key was fine right up until the file went somewhere the key shouldn’t, and config files go places.

I didn’t want go-tool-base handing every tool built on it that same slow-motion liability by default. So credential handling got rebuilt around a simple idea: the config file should usually hold a reference to the secret, not the secret itself.

Three modes, and which one you get

go-tool-base supports three ways to store a credential.

Environment-variable reference, the default. The config records the name of an environment variable, not its value:

anthropic:
 api:
 env: ANTHROPIC_API_KEY

The secret itself lives in your shell profile, your direnv setup, or your CI platform’s secret store, wherever you already keep that sort of thing. The config file now contains nothing sensitive at all. You can commit it, back it up, paste it into a bug report. The reference is inert on its own.

OS keychain, opt-in. The config holds a <service>/<account> reference and the actual secret goes into the operating system’s keychain: macOS Keychain, GNOME Keyring or KWallet via the Secret Service, Windows Credential Manager.

anthropic:
 api:
 keychain: mytool/anthropic.api

This one is opt-in by design, because the keychain backend carries dependencies that some deployments simply aren’t allowed to ship. (That opt-in mechanism turned out to be an interesting little problem all of its own, and it gets its own post in a couple of days.)

Literal value, legacy and grudging. The old behaviour. The secret sits in the config in plaintext:

anthropic:
 api:
 key: sk-ant-...

It still works, because breaking every existing tool’s config on an upgrade would be its own kind of vandalism. But it’s the last resort, it’s documented as the last resort, and the setup wizard puts a warning in front of you when you pick it.

The one place literal mode is not allowed

There’s a single hard “no” in all of this. If go-tool-base detects it’s running in CI (CI=true, which every major CI platform sets) the setup flow will refuse to write a literal credential, and exits non-zero.

The reasoning is that a plaintext secret written during a CI run is a plaintext secret written onto an ephemeral, often shared, frequently-logged machine, by an automated process that no human is watching. That’s the exact situation where the slow-motion liability becomes a fast one. CI environments inject secrets as environment variables already; there’s no good reason for a tool to be writing one to disk there, so go-tool-base simply won’t.

How it decides at runtime

A credential can be configured more than one way at once. You might have an env reference and an old literal key still lurking. So resolution follows a fixed precedence, highest to lowest:

The *.env reference. If that env var is set, use it.
Otherwise the *.keychain reference. If a keychain entry resolves, use it.
Otherwise the literal *.key / *.value, the legacy path.
Otherwise a well-known fallback env var (ANTHROPIC_API_KEY and friends), so a tool still picks up the ecosystem-standard variable with no config at all.

The useful property here is that adding a more secure mode transparently wins. Drop an env reference next to an old literal key and the next run uses the env var. You can migrate a credential to a better home without first removing it from its worse one, which makes the migration safe to do incrementally instead of as one nervous big-bang edit.

The tool tells on itself

A precedence rule is no use if nobody knows their config still has a plaintext key three layers down. So the built-in doctor command grew a check for exactly that. Run doctor, and if any literal credential is sitting in your config it reports a warning, names the offending keys (the key names, never the values) and points you at how to migrate.

It’s not an error. Literal mode is still legal. But the tool will quietly keep reminding you that you left the campsite messier than you could have, until you go and tidy it. (Old Scout habits die hard, and they’ve leaked all the way into the framework.)

The gist

A CLI tool that writes your API key into a plaintext config file isn’t doing anything wrong, exactly. It’s just handing you a liability that activates later, when the file travels somewhere the key shouldn’t. go-tool-base’s answer is three storage modes: an env-var reference by default, the OS keychain on request, and a plaintext literal only as a documented last resort that CI environments can’t use at all. Runtime resolution runs in a fixed precedence so a more secure mode always wins, which makes migrating a credential safe to do gradually. And doctor keeps an eye on the config so a stray plaintext secret doesn’t get to hide forever.

The secret should live in a secret store. The config file should just know its name.

A configurable AI endpoint is an attack surface

Sun, 19 Apr 2026 00:00:00 +0000

“Let users point at their own AI endpoint” is one of those config options that looks completely harmless on the way in. People want it, for perfectly good reasons. Then you sit with it for a minute and realise you’ve handed every user a loaded gun and pointed it vaguely at their own API key.

Why you offer it at all

There are real reasons to let someone set a custom base URL. They’re running a local model and want localhost:11434. They’re behind a corporate proxy that fronts the real provider. They’re on Azure’s flavour of OpenAI, which lives at a different host. They’ve a self-hosted gateway doing rate-limiting. All reasonable, all things a framework should support rather than fight.

The bit that’s a loaded gun

Here’s what the config option quietly decides: the base URL is where your credential goes. The API key rides along in an Authorization header on every request, to whatever host that URL resolves to. So the moment the endpoint is user-configurable, the destination of your secret is user-configurable too.

And users do user things. They paste a URL from a gist that turned out to be a honeypot. They leave http:// on the front, so the key crosses the wire in plaintext. They copy https://user:token@host/v1 not realising the userinfo changes who they actually authenticate to. They never edit the https://api.example.com/v1 placeholder and wonder why the key’s been posted to a domain they don’t own. None of that is malice. It’s what happens when the destination of a secret is a free-text field.

Validate before the first byte leaves

So every chat.New routes through ValidateBaseURL before the provider is built. The threat model is written at the top of pkg/chat/baseurl.go: an operator who can influence config could “redirect chat-provider traffic to an attacker-controlled HTTPS host and capture the Authorization header.” The checks run cheapest-first: a length cap, no ASCII control characters, must parse, no userinfo, https only, a host must be present, and the host mustn’t be a placeholder.

The userinfo rule is the sharp one:

if parsed.User != nil {
	// Reject any userinfo, with or without password. Never log
	// the URL itself because it contains the credential.
	return errors.WithHint(ErrInvalidBaseURL,
		"base URL must not contain credentials; use the Token field instead")
}

The placeholder check rejects example.com and friends and any subdomain of them, so the unedited https://api.example.com/v1 from a setup wizard never reaches the wire and hits some typosquatted lookalike. And the HTTP escape hatch is test-only by construction: the AllowInsecureBaseURL field that permits plain http is tagged json:"-", so a config file physically cannot set it. This all came out of the 2026-04-17 security audit, finding M-3.

rust-tool-base enforces the same at its own boundary: validate_base_url rejects userinfo, any scheme but https (bar a test-only allow_insecure), and documentation placeholder hosts like example.com.

What it can and can’t do

It won’t stop a user who deliberately points the tool at a malicious HTTPS host they genuinely chose. If someone is set on sending their own key somewhere bad, validation can’t read their mind.

What it stops is the accidents: the plaintext slip, the userinfo confusion, the placeholder nobody changed. Those aren’t theoretical, they’re the ones that happen to careful people on ordinary days. Storing the key well is one job (where a CLI keeps it), stopping it leaking through a log is another, and this is the third side of the triangle: once you’ve stored it and stopped it leaking, make sure you don’t send it somewhere daft.

Redacting the secret you didn't know was in the string

Sat, 18 Apr 2026 00:00:00 +0000

Dammit! How did that get there?

A log line that should never have existed. Not a password I’d carelessly printed, nothing as obvious as that. An upstream API handed me back an error, and it had quoted my own bearer token inside the message, and that error went straight into the logs the way errors do. I didn’t put the secret there. The error did. And I’d never have caught it by being careful, because being careful only protects you from the secrets you know you’re handling.

The easy half of redaction

Hiding the secrets you know about is the part everyone does. You’ve got an API key field, a password flag, so you mask them at the point you print them. key=****. Done, and it feels like you’ve solved redaction, when really you’ve solved the half that was never going to bite you.

The half that bites

The secrets that escape are the ones that arrive inside strings you don’t control. An upstream service echoes your token back in a 401 body. A connection string with the password in the userinfo, https://user:pass@host, lands in a debug line. A library stringifies a whole request, headers and all, for a “helpful” trace. You cannot field-mask a secret you didn’t know was in the string, because you never watched it go in.

You can’t register a value you never had, so match the shape

This is the bit I got wrong in my own head at first. I assumed redaction meant handing it the secrets I was holding so it could watch for them. But the dangerous secrets are exactly the ones I’m not holding a copy of. So pkg/redact doesn’t keep a registry of your values at all. It knows what secrets look like.

pkg/redact/redact.go carries a set of RE2 patterns: a credential in URL userinfo, an Authorization: header sitting in free text, query-string credentials, and the well-known provider prefixes:

prefixPatterns = []*regexp.Regexp{
	regexp.MustCompile(`sk-[A-Za-z0-9_\-]{16,}`), // OpenAI / Anthropic-style
	regexp.MustCompile(`ghp_[A-Za-z0-9]{30,}`), // GitHub PAT classic
	regexp.MustCompile(`github_pat_[A-Za-z0-9_]{30,}`), // GitHub fine-grained PAT
	regexp.MustCompile(`xox[baprs]-[A-Za-z0-9-]{10,}`), // Slack
	regexp.MustCompile(`AIza[A-Za-z0-9_\-]{30,}`), // Google API key
	regexp.MustCompile(`AKIA[A-Z0-9]{16}`), // AWS access key ID
}

Run any string through redact.String and an OpenAI key, a GitHub token or an AWS access key ID gets caught wherever it’s hiding, in an error you didn’t write, in a URL, in a stack trace, because each has a recognisable shape. For the secrets that don’t announce themselves with a prefix there’s a fuzzy fallback: any opaque alphanumeric run of 41 characters or more. The 41 is chosen on purpose, to clear UUIDs (36), MD5 (32) and git SHA-1 (40) without flagging them, while accepting that a SHA-256 (64) will trip it. A deliberate, documented trade rather than a magic number.

Where it runs

At the boundary where a string leaves for somewhere you can’t reach back into. The telemetry backend runs every event argument and error message through redact.String before it emits anything (pkg/telemetry/telemetry.go), and both telemetry and HTTP logging drop the value of any header redact flags as sensitive. It doesn’t matter which code path produced the string, or whether you even wrote that path; everything goes through the same gate and gets the same scrub.

rust-tool-base’s rtb-redact crate takes the same shape-matching approach: regex patterns, the same family of well-known provider prefixes, and an is_sensitive_header check for header values.

A realistic limit

It isn’t a force field. A secret with no recognisable shape, shorter than the fallback threshold, will sail through. You cannot redact what you cannot recognise. But the leak that actually keeps happening isn’t some exotic unknown, it’s a well-known token turning up in a place you didn’t expect, and a shape-matcher sitting at the edge catches exactly that, including secrets you never told it about. Which is the one thing registering your own values could never have done. Storing the key safely is a separate job, where a CLI keeps it; this is about making sure that, having stored it, it doesn’t quietly fall out through a log.

I had the framework audited: every finding was the same shape

Fri, 17 Apr 2026 00:00:00 +0000

When a real security audit lands back in your inbox, the temptation is to read it as a shopping list of unrelated mistakes. Fix one, fix the next, tick them off, move on. I did exactly that the first time. The second time, I noticed something far more useful: the findings weren’t scattered at all. They clustered. Almost every one was the same sentence with the nouns swapped out.

Findings cluster, they don’t scatter

When you get a real security audit back, the instinct is to read it as a list of unrelated mistakes. Finding 1, unrelated to Finding 2, unrelated to Finding 3. Triage each, fix each, move on.

That’s not what the go-tool-base audits looked like once I stopped reading them as a list. The findings clustered. Strip away the specifics and almost every one was the same sentence with the nouns swapped: untrusted input reaches a powerful operation, and nothing checks it in between.

That reframe is worth more than any individual fix, because it turns “we patched some bugs” into “we know where to look next time”. A framework’s attack surface isn’t spread evenly. It’s concentrated at the boundaries: the handful of points where data from outside (a config file, a command-line flag, something typed into a TUI, an HTTP response) flows into machinery that can be made to misbehave. Audit the boundaries and you’ve audited most of the risk. Three examples make the pattern obvious.

Boundary one: a regex compiler

Somewhere in the tool, a user-supplied string gets compiled into a regular expression. A search pattern typed into the docs browser, a filter from a config file. Feeding user input to regexp.Compile feels harmless. It’s just pattern matching, after all.

It isn’t quite harmless. A regular expression is a tiny program, and some tiny programs are catastrophically slow. A pattern with the wrong kind of nested repetition can take exponential time to evaluate against a modestly hostile input. That’s the class of bug known as ReDoS. A user, or something feeding the user’s config, hands you a pathological pattern and your tool wedges, burning a whole core, on what looked for all the world like a search box.

The fix isn’t to ban user-supplied regexes. It’s to stop treating “compile this string” as free. go-tool-base routes any regex whose pattern came from outside the binary through a regexutil.CompileBounded helper. It caps the pattern length and puts a hard timeout on compilation. A pattern known at build time can still use plain regexp.MustCompile, because that isn’t a boundary, it’s a constant. The discipline only applies where the input genuinely crosses in.

Boundary two: a URL opener

The tool needs to open a URL in the user’s browser, a docs link or an OAuth flow. Under the hood that’s the OS handler: xdg-open, or open, or rundll32.

Now ask where the URL came from. If any part of it is influenced by config, by a server response, by user input, then “open this URL” has quietly become “ask the operating system to do something with an attacker-influenced string”. A file:// URL. A javascript: URL. Something with control characters smuggled into it. The browser-open was never the dangerous part. The unvalidated string was.

So go-tool-base funnels every URL-open through one package, pkg/browser, and that package is a gate. It enforces an allowlist of schemes (https, http, mailto, and nothing else), bounds the length, and rejects control characters before the OS ever sees the string. The rule that makes it stick is that nothing else is allowed to call the OS handler directly. One door, and the door has a lock. A scattered capability with no chokepoint can’t be secured; a capability that has a chokepoint can. (You’ll have spotted the “one door out” idea by now… it’s the same instinct as the single error handler, pointed at security instead of consistency.)

Boundary three: a log sink

This one’s the sneakiest, because it runs the wrong way round. The first two boundaries are about dangerous input coming in. This one is about sensitive data leaking out.

The tool handles credentials. It also logs, emits telemetry, and reports errors, and all three of those are exit boundaries: places where strings leave the process for somewhere more persistent and more public, like a log aggregator, an analytics backend, an error tracker. If a token ever ends up in a string that flows to one of those, you haven’t logged an event, you’ve published a secret.

The defence is pkg/redact. Any free-form string heading for an observability surface goes through it first, and it strips the usual suspects: credentials in URL userinfo, sensitive query parameters, Authorization headers, the well-known provider key prefixes (sk-, ghp_, AIza and friends), long opaque tokens. The places most likely to leak, command arguments and error messages in telemetry, get it applied automatically rather than relying on every caller to remember.

Same pattern as the other two. A boundary, and something standing on it checking what goes through.

The grunt work

None of these fixes is clever. There’s no exploit demo, no neat trick to show off. Bound a length. Check a scheme against an allowlist. Run a string through a redactor. The work was almost entirely in noticing the boundary existed, and then making sure everything routes through the one checked path instead of dotting raw calls all over the codebase.

That’s the actual lesson of a security audit, and it’s why the cluster reframe matters. The value wasn’t the dozen-or-so individual fixes. It was learning that the next risk will be at a boundary too, the next place untrusted input meets a powerful operation with nothing in between, and that the job is to find those points and put a single, mandatory, checked door on each.

To sum up

A security audit of a CLI framework reads like a list of unrelated bugs and isn’t one. go-tool-base’s findings nearly all reduced to the same shape: untrusted input reaching a powerful operation unchecked. A regex compiler that needed a length and time bound (regexutil.CompileBounded). A URL opener that needed a scheme allowlist and a single chokepoint (pkg/browser). Log and telemetry sinks that needed credentials redacted on the way out (pkg/redact).

The fixes were structural and dull, which is exactly right. Find your boundaries (config, flags, TUI input, network responses, log and telemetry sinks), give each one a single mandatory checked path, and you’ve spent your audit effort where the risk actually lives.

A mutex on a flag nobody writes twice

Thu, 16 Apr 2026 00:00:00 +0000

“Why is there a mutex around a boolean that only ever gets set once?”

It’s a fair question, and I’d half-asked it of myself before someone asked it of me. The answer turns out to be written, in as many words, in a code comment I’ve grown rather fond of.

The registry and its one-way latch

go-tool-base keeps a feature registry: the initialisers, sub-commands, flags and checks that each feature adds to the CLI. Features register themselves into it at startup, from init(), before main runs. Once everything’s wired, the framework calls SealRegistry() and the registry latches shut. Any Register call after that point panics, on purpose, because a sub-command or flag that turns up after the CLI has parsed its arguments is a bug I want to hear about at once, not discover three releases later.

So there’s a registrySealed bool. It starts false, SealRegistry flips it to true exactly once in normal operation, nothing flips it back outside of tests, and it’s read on every registration attempt. Written once, read many. The textbook shape of “you don’t need a lock for this.”

Except the comment disagrees, on purpose

Here is the actual declaration, in pkg/setup/registry.go:

// registryMu protects globalRegistry and registrySealed. Acquired for write
// by all Register* and Reset/Seal helpers; acquired for read by all Get*
// accessors. The mutex is required for memory visibility of registrySealed
// across goroutines, not only mutual exclusion on the maps.
var (
	registryMu sync.RWMutex
	registrySealed bool
)

That last sentence is the entire post. The mutex has an obvious day job: the registry is a clutch of maps that get appended to during registration, and concurrent appends need genuine mutual exclusion. registrySealed could have just hitched a ride on that lock and nobody would have thought twice. But the comment goes out of its way to say the lock is also required for the flag, for visibility, not only exclusion.

Why a write-once bool still needs the lock

The Go memory model makes no promise that a goroutine reading registrySealed will ever see the write SealRegistry made, unless there is a happens-before relationship between them. No synchronisation, no guarantee. A reader can sit there seeing false long after the seal happened on another goroutine, because the compiler may cache the read and the CPU may serve it from a core-local view. And a concurrent read and write of the same variable, with nothing ordering them, isn’t “probably fine”; it’s a data race, which Go defines as undefined behaviour.

“But registration is single-threaded, it’s all init().” It was, right up until we wanted the tests to run in parallel. This lock exists because of a deliberate campaign to restore t.Parallel() across the codebase after a stack of races forced us to drop it (the same campaign that retired the package-level mocking hooks). Tests build, register, seal and reset this registry from parallel goroutines. The instant that’s true, the seal check has to stay correct while racing, because the very thing it guards against is concurrency. So reads take registryMu.RLock, the write takes registryMu.Lock, and now there’s a happens-before edge: anyone who acquires the lock after SealRegistry released it is guaranteed to see true.

What the lock is actually for

It isn’t there to stop two goroutines both sealing the registry. There’s only ever the one seal. It’s there so that every reader can trust what it reads. A value written exactly once is precisely the case where you’re most tempted to skip the synchronisation, and precisely the case where skipping it can leave a reader legally staring at the stale value for good. The comment spells it out so that the next person to glance at registrySealed, think “that clearly doesn’t need a lock,” and reach for the delete key, reads the sentence first.

(There’s a sibling sealed flag in the middleware registry that follows the identical pattern, for the identical reason.)

The test-mocking pattern that races

Thu, 16 Apr 2026 00:00:00 +0000

I’m going to tell you about a bug go-tool-base shipped, because it’s one of those bugs that’s so reasonable-looking you’ll find it in textbooks, conference talks, and an awful lot of otherwise excellent Go code. We had it too. It passed every test on my laptop, every single time, and then quietly fell over on CI while blaming an innocent bystander.

It’s the classic Go trick for mocking a dependency, and it races.

A pattern that looks completely reasonable

Here’s a thing you need to do constantly in Go tests: stop a function from really shelling out. It calls exec.LookPath to find a binary, or exec.Command to run one, and your test very much does not want it touching the real $PATH or spawning a real process.

The Go community has a well-worn answer. Hoist the function into a package-level variable, call that, and let tests reassign it:

// production code
var execLookPath = exec.LookPath

func findTool() (string, error) {
 return execLookPath("sometool")
}

// test
func TestFindTool(t *testing.T) {
 old := execLookPath
 defer func() { execLookPath = old }()
 execLookPath = func(string) (string, error) {
 return "/fake/path", nil
 }
 // ...assert...
}

It’s tidy. No interface to thread through, no constructor to change. You’ll find it in a great deal of Go code, including some very respectable Go code indeed. go-tool-base had it too.

And it works. It works on your machine, it works in code review, it works the first hundred times CI runs it. Which is precisely what makes it dangerous, because it’s wrong, and it’s just been biding its time.

Add one line and it detonates

Go’s t.Parallel() is more or less free performance. Mark your tests with it and the runner overlaps them instead of plodding through one at a time. On a package with a few hundred tests it’s a real, worthwhile speed-up, so naturally you reach for it.

Now picture two tests, both using the pattern above, both marked t.Parallel(). They run concurrently. Test A assigns its fake to execLookPath. Test B assigns its fake to execLookPath. Test A reads execLookPath expecting its own fake. Two goroutines, one variable, writes and reads with nothing synchronising them. That’s a textbook data race, and the textbook is right: the behaviour is undefined. Test A might see B’s fake. The deferred restore might land in the wrong order and leave the variable pointing at a fake after both tests have finished, poisoning a third one for good measure.

The truly nasty part is the intermittency. Whether the race actually bites depends on goroutine scheduling, which depends on machine load and core count. Your laptop running eight tests at once might never lose the coin-toss. A CI runner under load, scheduling differently, loses it and fails a test that has nothing obviously to do with the change in the commit. You re-run the pipeline, it passes, everyone shrugs and moves on. A test suite that fails one run in twenty trains your team to ignore it, and an ignored CI failure is worse than no CI at all.

I can tell you this one from direct, slightly embarrassed experience, because go-tool-base shipped exactly this bug and CI caught it the honest way: green on the laptop, red on the runner, with the failure cheerfully pointing at innocent bystander tests rather than the global that was actually the culprit. go test -race will name it for you if you crank the parallelism up high enough to lose the toss reliably… but you have to go looking, and you only go looking once it’s already ruined an afternoon.

The fix isn’t synchronisation, it’s structure

The instinct is to slap a mutex around the variable. Resist it. A mutex makes the race defined, but it doesn’t make the design any good. You’ve still got global mutable state, you’ve just queued the fight instead of cancelling it. And tests that serialise on a shared lock aren’t really parallel any more, so you’ve also handed back the speed-up you came for in the first place.

The real fix is to not have a shared variable at all. The dependency was always an input to the code; the package-level var was just a way of avoiding saying so out loud. So say it. Inject it.

A struct field:

type Finder struct {
 lookPath func(string) (string, error) // defaults to exec.LookPath
}

func (f *Finder) find() (string, error) {
 return f.lookPath("sometool")
}

Or a functional option, if you’d rather keep the zero value clean. Either way, each test constructs its own Finder with its own fake. There’s no shared variable, so there’s no race, and t.Parallel() is free again because the tests genuinely don’t touch each other.

go-tool-base wrote this straight into its standing rules: no package-level mocking hooks, full stop. Dependencies come in through struct fields, functional options, or config fields. (The same injection discipline that makes Props so testable, applied one rung further down.) And to stop everyone hand-rolling the same exec fakes, there’s a small internal package, internal/exectest, with ready-made LookPath and CommandContext doubles you construct per-test. The pattern is gone, and the door it came in through is shut.

The rule worth taking away

A package-level variable that tests reassign is shared mutable state. It reads as a harmless convenience because in a single-threaded test run it behaves like one. t.Parallel() is the thing that reveals it was never harmless, only unobserved.

The general lesson is older than Go: if a value is an input to your code, make it an input. Smuggling it in as a global is borrowing test-time convenience against a debt that comes due, with interest, the day someone wants their tests to run in parallel. Pay cash. Inject the dependency.

Worth remembering

Mocking via a reassignable package-level variable is a beloved Go shortcut and a latent data race. It survives because single-threaded test runs hide it; t.Parallel() exposes it as intermittent, bystander-blaming CI flake that’s miserable to trace. A mutex only makes the bad design defined. The fix is structural: inject the dependency as a struct field or functional option, so each test owns its own double and there’s no shared state to race over. go-tool-base banned the global-hook pattern outright and ships internal/exectest so nobody’s tempted back to it.

If a piece of code depends on something, let it say so in its signature. Your future self, staring at a CI failure that flatly refuses to reproduce, will thank you.

Testing code that calls an LLM: yes, you actually can

Wed, 08 Apr 2026 00:00:00 +0000

“You can’t test code that calls an AI.” I’ve heard it said with great confidence, and it’s half right, which is the most dangerous kind of right. You genuinely can’t assert on what a non-deterministic model says. But the model isn’t your code, and the bits sitting either side of it most certainly are.

“You can’t test AI code”

It’s a fair worry. Your command calls an LLM. The LLM returns something slightly different every run. A test that asserts response == "..." is broken before you’ve finished typing it. So the conclusion arrives quickly: the AI path can’t be tested, leave it uncovered.

Which is a shame, because the AI call is usually the riskiest line in the whole command.

The conclusion is also wrong. It mistakes “I can’t test the model” for “I can’t test my code”. The model is not your code. Your code is the two pieces sitting on either side of it.

Your code is a prompt and a handler

Strip the command down to what it actually does:

It builds a prompt. It assembles a system prompt, the user’s input, perhaps some context, and sends it.
The model does something. This is not your code.
It takes the response and does something with it. It parses it, branches on it, prints it, stores it.

Steps one and three are entirely yours, and entirely deterministic. The same inputs build the same prompt and handle the same response the same way, every single time. That’s testable. Step two is the only part that isn’t, and step two was never yours to test in the first place.

So the job is to pin step two to a known value, and then test one and three properly.

Test the prompt: snapshot it

Step one produces a prompt, and a prompt is just a string, which means you can pin it.

Both frameworks lean on snapshot testing here. go-tool-base uses a golden-file approach: the prompt your code generates is recorded to a file, and the test re-generates it and compares against that file. rust-tool-base does the same with insta, snapshotting the request body the client would send.

The reason this matters is that the prompt is load-bearing and quietly easy to break. You refactor how context gets assembled. Without noticing, you’ve changed the wording, or the ordering, or dropped a line the model was leaning on. Nothing fails to compile. The behaviour just drifts, silently.

A snapshot test catches exactly that. It fails, it shows you the diff between the old prompt and the new one, and it makes you stop and make a decision. Was this change intended? If yes, you accept the new snapshot and move on. If no, you’ve just caught a bug before it shipped. Either way the prompt never changes by accident, which for AI code is most of the battle.

Test the handler: mock the response

Step three needs a response to handle, and in a unit test you don’t get that response from the real model. You supply it.

go-tool-base ships generated mocks for the ChatClient interface. A test builds a mock client, tells it “when Ask is called, return this canned value”, and runs the command against it:

mockClient := mock_chat.NewMockChatClient(t)
mockClient.EXPECT().
 Ask(mock.Anything, mock.Anything, mock.AnythingOfType("*main.Analysis")).
 RunAndReturn(func(_ context.Context, _ string, target any) error {
 *(target.(*Analysis)) = Analysis{Severity: "critical"}
 return nil
 })

Because the interface is only four methods, that mock is trivial to set up and complete by construction. rust-tool-base takes the same idea one layer down: HTTP-bound tests use wiremock, which stands up a fake server returning a canned response body. The client makes a real HTTP request; it just goes to a fake endpoint the test controls.

Either way, step two is now fixed to a value you chose, which makes step three deterministic. And that unlocks the tests that actually matter: given a malformed response, does the command fail gracefully? Given a rate-limit error, an empty answer, a field missing? Those are the cases a live model almost never hands you on demand, and a mock hands you every time, on the first run.

This is, incidentally, the same discipline as the test-mocking work elsewhere in the framework: the dependency is injected, so the test gets to decide what it does.

What you deliberately don’t test

One boundary worth stating. None of this tests whether the model gives good answers. That question is real, but it’s a different activity (evaluations, run as their own suite) and not something to mix into the unit tests.

The unit suite’s job is your code: that it builds a sound prompt, and that it handles every shape of response correctly, including the ugly ones. Keep that well away from “is the model clever today”. A unit test that depends on the model being clever is a unit test that fails when the weather changes, and a flaky test just teaches people to ignore the whole suite.

What it comes down to

Code that calls an LLM is testable; the model is not, and those are different statements. Your code is a prompt builder and a response handler, both deterministic, with the model sat in between.

go-tool-base and rust-tool-base converge on the same approach. Snapshot the prompt, with golden files or insta, so a refactor can’t change what you send without a test noticing. Mock the response, with generated ChatClient mocks or a wiremock server, so tests run with no network and you can feed in the malformed and error cases a real model won’t reliably produce. Leave “are the answers any good” to a separate evaluation suite. Test the two halves you own, and the non-determinism in the middle stops being an excuse to leave the riskiest line uncovered.

The AI provider that isn't an API

Mon, 06 Apr 2026 00:00:00 +0000

go-tool-base’s chat package puts five AI providers behind one interface. Four of them are exactly what you’d guess: HTTP calls to OpenAI, Claude, Gemini, and anything OpenAI-compatible. The fifth one isn’t an API at all. It shells out to a binary.

That sounds like a slightly mad thing to want, right up until you’ve worked somewhere the network says no.

The fifth provider shells out

The chat package speaks to five providers through one ChatClient interface. Four of them are what you’d expect: HTTP requests to OpenAI, to Claude, to Gemini, to any OpenAI-compatible endpoint. The tool author picks one in config, and the rest of the code never knows the difference.

The fifth, ProviderClaudeLocal, is different in kind. It doesn’t make an HTTP request at all. It shells out. It runs the claude CLI binary as a child process, passes the prompt in, and reads the answer back from the binary’s output.

That sounds like an odd thing to want until you’ve been stuck in the environment it was built for.

Why you’d want that

Picture a corporate network with its egress locked right down. Outbound HTTPS to api.anthropic.com is blocked by policy. A tool built on go-tool-base that uses AI would simply fall over there. It tries to reach the API, there’s no route, and that’s the end of the feature.

But the developer at that machine has the claude CLI installed, and has run claude login. That binary is permitted. It’s an approved, managed tool, and it has its own sanctioned path out. The direct API call is blocked; the claude command is not.

ProviderClaudeLocal is what bridges those two facts. If your tool’s AI calls go through that already-blessed binary instead of straight at the API, they work, in an environment where the direct call cannot. That’s the whole reason the provider exists. It isn’t faster (a real API call has lower latency) and it isn’t more capable. It’s for the place where the API call simply isn’t an option, and “isn’t an option” is a surprisingly common place to find yourself inside a large organisation.

What it costs

It’s worth being straight about the trade, because ProviderClaudeLocal is the reduced-capability provider.

It doesn’t do tool calling. It doesn’t do parallel tools. It doesn’t stream. Those need a live, structured connection to the model’s API, and a subprocess that runs once and prints an answer is not that. What it does support is plain chat and structured output, the latter through the binary’s own --json-schema flag.

So the positioning, and the package’s documentation says exactly this, is: prefer the API providers when you can reach them, because they’re lower latency and feature-complete. Reach for ProviderClaudeLocal when API access is restricted. You accept the narrower capability set as the price of working at all. For a tool whose AI feature is “answer a question” or “return a structured analysis”, that price is often nothing you’d even notice. For one built on an agentic tool-calling loop, it’s a real limitation, and you’d know to expect it.

How it stays behind the same interface

Here’s the part that makes it pleasant rather than a special case to maintain. Despite being a subprocess and not an API, ProviderClaudeLocal is still a ChatClient. Your feature code calls Chat and Ask exactly the way it would for any other provider.

Everything that makes a subprocess provider awkward stays inside the provider. Spawning the binary, feeding it the prompt, parsing its output, capturing stderr and surfacing it when the binary exits non-zero, and threading multi-turn continuity through session identifiers passed back on the next call with --resume: all of that is the provider’s problem, and all of it sits behind the interface. The code in your tool that uses AI doesn’t know, and has no way to find out, that this particular provider is a child process rather than an HTTPS call.

That’s a unified interface genuinely earning its place. It’s easy to put a uniform face on four things that already work the same way underneath. The real test of the abstraction is whether something that works in a completely different way, a subprocess instead of a socket, can still slot in without the caller changing a line. Here it can. You swap one config value, and a tool that talked to an API now talks through a binary, and nothing downstream so much as blinks.

The bottom line

go-tool-base’s chat package puts five providers behind one ChatClient interface, and ProviderClaudeLocal is the one that isn’t an API. It runs the locally installed, pre-authenticated claude CLI as a subprocess.

It exists for the locked-down environment where outbound HTTPS to the AI API is blocked but the claude binary is allowed: there, AI features keep working where a direct call would fail. The trade is a narrower capability set (no tool calling, no streaming, plain chat and structured output only) so you prefer the API providers when you can reach them and fall back to this when you can’t. And because it’s still a ChatClient, all the subprocess machinery stays hidden, and your code uses it without knowing it’s there. That last part is the real test of an abstraction: a provider that works in an entirely different way still slots in unchanged.

AI conversations you can resume

Sat, 04 Apr 2026 00:00:00 +0000

An AI conversation is, fundamentally, its own history. The model’s next answer depends on everything said so far. And a CLI tool, by its very nature, forgets everything the moment it exits. Put those two facts together and you get the problem: run an AI command, exit, run it again, and you’re talking to someone who’s never met you.

A CLI forgets everything

A long-running service keeps its state in memory for as long as it runs. A CLI tool doesn’t get that luxury. It starts, does one thing, exits. The next invocation is a brand-new process with no memory of the last one.

For most commands that’s exactly right, and you wouldn’t want it any other way. But an AI conversation is a different kind of beast, because a conversation is its history. The model’s next answer depends on everything said so far. Run an AI command, exit, run it again, and you’ve started a fresh conversation with someone who’s never met you. For an interactive assistant, or any AI workflow that unfolds across several invocations, that’s plainly the wrong behaviour. The user expects to pick up where they left off.

Save and restore

The chat package handles this through a PersistentChatClient interface. Like streaming, it’s an optional capability discovered with a type assertion, sitting beside the four-method core rather than bloating it. A client that supports persistence also satisfies this interface:

if pc, ok := client.(chat.PersistentChatClient); ok {
 snapshot, err := pc.Save()
 // store the snapshot somewhere
}

A snapshot is a serialisable value that captures the conversation. You store it. Next run, you load it, Restore it onto a fresh client, re-register your tools, and call Chat again. “Where were we?” works, because the model is handed back the whole history.

A snapshot is opinionated about what it carries

The interesting part is what a snapshot does and doesn’t contain, because that’s a series of deliberate decisions.

It carries the messages, the system prompt, the model name, and tool metadata: the names, descriptions and parameter schemas of the tools that were registered.

It does not carry tool handlers. Handlers are code, not data; you can’t serialise a function meaningfully, so after a restore you re-register them with SetTools. The snapshot remembers that a tool called read_file existed and what its shape was; it doesn’t try to remember the Go function behind it.

And it does not carry API tokens. This is the one to dwell on. A snapshot is a file. A file gets synced, backed up, copied between machines, attached to a support ticket by a user trying to be helpful. A snapshot that carried the API key would be a credential leak the moment it left the laptop it was made on. So the snapshot never contains a token, at all. On restore, the client picks the credential up again the ordinary way, from the environment or the keychain. The conversation and the secret are kept in separate places on purpose, and only one of them is ever in the file.

Encrypted at rest, if you want it

The package ships a FileStore that writes snapshots as JSON files, with 0600 permissions in a 0700 directory, and it can encrypt them. Pass WithEncryption a 32-byte key and snapshots are written with AES-256-GCM.

That option exists because a conversation can hold sensitive content even when it holds no credential. The log a user pasted in for analysis, the source file they asked the model to review, the internal details tucked into their questions: none of that is an API key, and all of it might be something you’d rather not have sitting in plain JSON in a backup somewhere. Encryption at rest covers it.

The FileStore is also careful about the snapshot identifiers it’s handed. An ID has to be a canonical UUID, and the resolved file path is checked to lie inside the store directory, so a snapshot ID arriving from an untrusted source (a CLI flag, a request payload) can’t be bent into a path-traversal that reads or writes somewhere it shouldn’t. Persisting conversations adds a small filesystem surface, and the store treats it as exactly that.

The short version

A CLI tool forgets everything between invocations, which is correct for most commands and wrong for an AI conversation, because a conversation is its history.

go-tool-base’s chat package lets you persist one. PersistentChatClient saves a snapshot you can store and restore later, picking the conversation back up where it ended. The snapshot is deliberate about its contents: messages, system prompt and tool metadata yes; tool handlers no, because they’re code you re-register; API tokens never, because a snapshot is a file and a file travels. The built-in FileStore can encrypt snapshots at rest with AES-256-GCM and validates snapshot IDs against path traversal. Resumable conversations, without the conversation file turning into a place secrets leak from.

An AI agent that has to make the build pass

Thu, 02 Apr 2026 00:00:00 +0000

Most AI code generation works on a charming little principle I’ll call generate-and-hope. The model writes the code, the model stops at the closing brace, and whether the thing actually compiles is left as an exercise for you. For a snippet you paste into an editor, fine. For a whole generated command, that’s just outsourcing the disappointment.

go-tool-base does something I’m rather happier with: the AI has to make the build pass before it’s allowed to claim it’s done.

Generate and hope

The usual shape of AI code generation is this. You ask for code, the model produces it, and the model’s job ends at the closing brace. Whether it compiles, whether the tests pass, whether the imports even resolve, none of that has been checked. The model produced something that looks right. You find out whether it is right when you build it.

For a snippet you paste into an editor, that’s perfectly fine. The compiler tells you in a second. But go-tool-base’s generator, driven by gtb generate command --script or --prompt, produces a whole command: the implementation, its tests, the lot. “Generate and hope” at that scale means handing the user a project that may or may not build, and quietly making them the one who finds out which.

Drafting is only step one

So the generator doesn’t stop at drafting. Writing the first version of the implementation and its tests is step one of two. Step two is an autonomous repair agent.

Once the draft is on the filesystem, a separate agent takes over. It’s an LLM running in a loop, but a loop aimed at one narrow, checkable job: make this project build and pass its tests. It isn’t asked to be creative. It’s asked to get to green.

A fixed set of tools, and no shell

The agent is not handed a shell. It’s given a fixed, defined set of tools and nothing else. Three of them let it explore and edit the project: list_dir, read_file, write_file. Four of them let it verify the project:

go_build runs the build and captures the compiler errors.
go_test runs the tests and captures the failures.
go_get resolves a missing dependency.
golangci_lint runs the project’s linter.

That restriction is the design, not a limitation of it. The agent can’t delete arbitrary files, can’t reach the network, can’t run anything that isn’t on the list. It has exactly what it needs to make code compile and nothing it would need to do damage. Its file writes are confined to the project directory by an explicit path check, so even write_file can’t go wandering up into /etc. A coding agent you’d actually let near a filesystem is one whose abilities are an allowlist, not a denylist. (I keep coming back to that principle through this series… safety as a boundary you draw, not a behaviour you hope for.)

The loop

The repair loop is a ReAct loop, the same reason-act-observe shape as the tool-calling loop, only this time pointed at a goal:

The draft is on disk.
Verify: run go_build and go_test.
If verification failed, read the error logs, the compiler error or the failing test.
Reason about the cause: an undefined variable, a missing import, a wrong signature.
Act: call write_file to patch the code, or go_get to add the dependency.
Loop. Steps two to five repeat until the project is green, or the agent hits its bounded step limit.

What makes this work is treating the error output as feedback rather than as a failure to log and walk away from. A compiler error is the single most useful sentence you can hand a model that’s trying to fix code. It says what’s wrong, and usually where. The loop feeds it straight back in, and the model fixes against it.

Verification changes what “done” means

Here’s the real shift, and the agent’s own documentation puts it well: the agent “doesn’t just say it fixed a bug; it uses a Test tool to verify the fix before reporting success.”

A generate-and-hope model reports success when it finishes writing. It has no idea whether the code works, and it isn’t really claiming otherwise. “Done” means “I produced text”. The repair agent reports success when go_build and go_test actually pass. “Done” means “the build is green”. Those are two completely different claims, and only the second is worth anything to the person who asked for the command.

That’s the line between an AI that’s a creative writer and an AI that’s a collaborator you can hand a task to. And when the agent can’t reach green, when it spends its whole step budget and the project is still broken, the generator fails safely: it leaves the best-attempt code in place, commented out so the project still compiles, and tells the user what to finish by hand. There’s also an --agentless flag for anyone who’d rather have a plain single-shot retry than the multi-step agent. The default, though, is the agent, because the default should be code that’s been checked.

Where this leaves us

Most AI code generation generates and hopes: the model writes code and the user discovers whether it works. For a whole generated command, that pushes a may-or-may-not-build project onto the user.

go-tool-base’s generator drafts the command and then hands it to an autonomous repair agent. The agent has a fixed set of tools (explore and edit the project, build it, test it, lint it, fetch dependencies) and no shell at all, with file writes confined to the project directory. It runs a ReAct loop, reading each error and patching against it, until the build is green or it exhausts its steps. The point is what “done” comes to mean: not “the model finished writing”, but “the build passes”. Only one of those is a claim worth trusting.

Stop regex-ing the LLM's prose

Tue, 31 Mar 2026 00:00:00 +0000

Ask an LLM a question and it hands you back prose. Lovely to read, miserable to program against. You wanted the one number buried in the middle of it, and now you’re writing a regular expression to fish a word out of three well-written paragraphs that phrase themselves slightly differently every single time you run them.

There’s a much better way, and it’s the difference between forever interpreting an LLM and actually building on one.

The problem with a paragraph

You ask an LLM to analyse a log file and tell you the severity of what it found and a suggested fix. It comes back with three well-written paragraphs. Somewhere in there is the word “critical”, and somewhere is the fix.

Your program now has to extract those two facts from prose, and prose has no contract. The next run, the model phrases it differently. It leads with a caveat. It says “severe” where last time it said “critical”. It puts the fix first. Anything that worked by finding “critical” in the text is now quietly wrong, and you didn’t change a line. Parsing free text for structured facts is a game you lose slowly.

What you actually wanted was never a paragraph. It was a value: a thing with a severity field and a fix field, that you can branch on and store and pass around like any other.

Ask for the struct, not the prose

go-tool-base’s chat package draws the line with two methods. Chat gives you text. Ask gives you a struct.

You define the Go type you want back:

type Analysis struct {
 Severity string `json:"severity"`
 Fix string `json:"fix"`
}

var result Analysis
err := client.Ask(ctx, "Analyse this log file: "+logText, &result)

The framework generates a JSON Schema from that struct, sends it to the model as the required response format, and unmarshals the reply straight into result. You never lay a finger on the prose. You get result.Severity and result.Fix, typed, ready to use. If you want the model’s answer to drive a switch statement, this is the method that lets it.

The struct is the schema is the contract

The detail that makes this hold up over time: you don’t write the schema. The struct is the schema.

The framework derives the JSON Schema from your type. In go-tool-base that’s GenerateSchema[T](); in rust-tool-base the schema comes from your Rust type through schemars. (Yes, there’s a Rust sibling now. I’ll introduce it properly in a few weeks, but it keeps gatecrashing these posts because the two frameworks deliberately share ideas.) Either way there’s one definition, your type, and the schema is just a projection of it.

That matters, because otherwise two things have to agree. There’s the schema you tell the model to obey, and there’s the type you unmarshal the answer into. Hand-write the schema and those two can drift: add a field to the struct, forget to add it to the schema, and the model is never told to produce it, so it silently never appears. Deriving the schema from the type collapses the two into one. They can’t disagree, because there’s only one of them.

Both frameworks, with one extra step in Rust

go-tool-base does this with Ask and a ResponseSchema set on the client config. rust-tool-base does it with chat_structured::<T>, where T is any type that’s both deserialisable and JsonSchema.

rust-tool-base adds one step worth calling out. Before it deserialises the model’s reply into your T, it validates the raw response against the schema with a JSON Schema validator. That splits the failure into two distinct, named cases: the response didn’t match the schema, or it matched the schema but still wouldn’t deserialise. A model that returns subtly wrong JSON fails loudly and specifically, with an error that tells you which of those happened, instead of quietly handing you a zero-valued struct that you end up debugging an hour later.

When you’d reach for it

The line is simple, and it’s about who reads the answer.

If a human reads the answer, prose is right. Chat, free text, let the model write well. A summary, an explanation, an interactive reply: leave all of those as prose.

If a program consumes the answer, you want a value. Classification, extraction, a code review scored out of a hundred with a list of issues, a yes-or-no with reasons: anything where the next thing that happens is your code branching on the result. There, Ask and chat_structured turn the LLM from something you have to interpret into something that returns a value, and a typed value is a thing you can actually build on.

To sum up

An LLM returns prose by default, and prose has no contract, so a program that picks structured facts out of it breaks the moment the model rephrases.

Structured output asks for the value instead. You define a struct, the framework derives a JSON Schema from it, the model is constrained to that shape, and you get a typed result. go-tool-base’s Ask and rust-tool-base’s chat_structured both work this way, with the schema derived from your type so the schema and the type can’t drift; rust-tool-base additionally validates the response against the schema before deserialising. Use it whenever the answer feeds code rather than a human. It’s one of the four methods that make up go-tool-base’s small chat interface, and it’s the one that makes an LLM safe to program against.

Telemetry that asks first

Mon, 30 Mar 2026 00:00:00 +0000

Usage telemetry is genuinely useful. Knowing which commands people actually run, where the errors cluster, whether anyone ever touched the feature you spent a fortnight on… that’s the stuff that makes you a better maintainer. Wanting it is completely legitimate.

The trouble is that the usual way of getting it, on by default and quietly hoovering up everything, is a small betrayal of the people who installed your tool to get a job done. I wasn’t willing to build that, so go-tool-base’s telemetry starts from a different question.

The data you want, and the line you shouldn’t cross

If you maintain a tool, you want to know how it’s actually used. Which commands matter and which are dead weight. Where the error rate spikes. Whether anyone touched the feature you spent that fortnight on. That information makes you a better maintainer, and, to say it again, wanting it is completely legitimate.

The trouble is the standard way of getting it. Telemetry on by default. An opt-out buried three levels down in a settings file nobody reads. And once it’s running, it quietly collects far more than it ever admitted to: the arguments people passed, the paths they were working in, an IP address for good measure.

Every one of those is a small betrayal of someone who installed your tool to get a job done, not to become a data point. And the cost when users notice isn’t a slap on the wrist. It’s trust, and trust in a developer tool does not grow back quickly. A tool that surprises you once with what it was quietly collecting is a tool you uninstall and warn your colleagues about.

So go-tool-base’s telemetry started from a different question. Not “how do we collect the most data” but “how do we collect useful data without ever putting the user in a position they didn’t choose”.

Rule one: it is off until you say otherwise

The foundation is the simplest possible rule, and it’s absolute. Telemetry is never enabled by default. A freshly installed tool built on go-tool-base sends nothing. Not a heartbeat, not a ping, nothing at all.

It only starts collecting when the user makes an explicit, visible choice to let it. Three honest doors: they run telemetry enable, they say yes to a clear prompt during init, or they set TELEMETRY_ENABLED themselves. All three are deliberate acts. None of them is a pre-ticked box or a default they have to discover and then undo.

This is opt-in, and the distinction from a well-hidden opt-out is the entire point. Opt-out telemetry treats consent as something to be assumed and grudgingly reversed. Opt-in treats it as something that has to be given. Only one of those is actually consent.

Rule two: no personally identifiable information, full stop

Consent to “some telemetry” is not consent to “any telemetry”, so the second rule constrains what can ever be collected, even from a user who’s opted in.

No personally identifiable information. The framework does not record command arguments (they routinely contain paths, hostnames, the occasional secret someone’s pasted in). It does not record file contents. It does not record IP addresses.

It does need some notion of “distinct installations” for the numbers to mean anything, so it derives a machine ID from a handful of system signals and runs it through SHA-256. What leaves the machine is a hash. It tells you “this is the same install as last week” and tells you precisely nothing about whose install it is, and the hash can’t be walked backwards into the signals it came from.

The events themselves are deliberately thin. Which command ran, roughly how long it took, whether it errored. The shape of usage, not a transcript of it.

Rule three: the author picks the destination

Even with consent given and PII excluded, there’s a third question: where does the data actually go? go-tool-base doesn’t answer that for you, because it can’t. A corporate internal tool, an open-source CLI and an air-gapped utility have completely different right answers.

So the backend is the tool author’s choice. The framework ships several (a noop backend, stdout, a file, plain HTTP, and OpenTelemetry over OTLP) and supports custom ones. The noop backend matters more than it looks: it lets a tool wire up the whole telemetry surface, commands and all, while sending data precisely nowhere. A perfectly reasonable, fully supported configuration.

Pluggable backends also mean the data never has to touch any infrastructure I run. It goes where the tool’s author decides, on their terms. The framework provides the plumbing and stays well out of the destination.

And a way back out

One last thing, because it’s the part that makes the opt-in real rather than decorative. A user who opted in can opt straight back out, and the package includes a GDPR-aligned deletion path, so “stop, and remove what you have” is an actual supported request rather than a polite fiction.

Consent you can’t withdraw isn’t consent. It’s a one-way door with a friendly sign on it. The deletion path is what keeps the front door an actual door.

The bottom line

Telemetry is genuinely useful to a maintainer and genuinely dangerous to the trust of the people running the tool, and the usual implementation (on by default, opt-out buried, collecting everything) spends that trust recklessly. go-tool-base’s telemetry holds three lines: never enabled without an explicit user action, never collecting personally identifiable information even once enabled, and always sending data to a destination the tool’s author chose, up to and including nowhere. A real deletion path makes the opt-in something you can take back.

You can have your usage numbers. You just have to ask for them, the way you would for anything else that wasn’t yours to begin with.

Letting the AI call your Go functions

Sun, 29 Mar 2026 00:00:00 +0000

An AI that can only produce text can describe your system. An AI that can call your Go functions can actually operate it. That gap, between describing and doing, is the difference between a chatbot and something genuinely useful, and crossing it comes down to one fiddly mechanism: tool-calling, and the loop that drives it.

Talking about the system versus operating it

Wire an AI provider into a CLI command and you get something that can talk. Ask it a question, get a paragraph back. Useful, up to a point.

But notice the ceiling. An AI that can only generate text can describe things. It can tell you what it would do. What it can’t do is look at the actual current state of your system, or take a real action, because it has no hands. It’s reasoning in a vacuum about a world it can’t reach out and touch.

The thing that gives it hands is tool-calling. You hand the AI a set of functions it’s allowed to call. Now, mid-conversation, it can decide it needs to read that file before it can answer, or run that query, or check that status, and actually go and do it, and then reason about the real result. The AI stops describing your system and starts operating it.

The loop is the hard part

Tool-calling has a shape, and the shape is a loop. The literature calls it ReAct: Reason, Act, Observe.

The AI reasons about the prompt and decides whether it needs a tool.
If it does, it acts, asking for a specific tool with specific arguments.
Your code runs the tool and feeds the result back. The AI observes that result.
Round again. Reason about the new information, maybe call another tool, maybe several. Keep going until the AI has what it needs and produces a final text answer with no more tool calls.

Conceptually simple. Tedious and error-prone to implement by hand every single time: parsing the model’s tool-call requests, dispatching to the right function, marshalling arguments in and results out, feeding observations back in the exact format the provider expects, knowing when to stop, and not looping forever if the model gets itself stuck.

That orchestration is pure plumbing, and it’s identical for every tool and every command. So you can probably guess what’s coming: go-tool-base’s chat package owns it. You don’t write the loop. You write the tools.

Defining a tool

A chat.Tool is four things: a name, a description, a parameter schema, and a handler. The description is what the AI reads to decide whether to use the tool, so it’s worth writing well. The schema describes the arguments, and you don’t hand-write it. You write a tagged Go struct and let it generate:

type ReadFileParams struct {
 Path string `json:"path" jsonschema_description:"Relative path to the file"`
}

The struct is the contract. The framework derives the JSON Schema the AI is given straight from those tags, so the schema and the Go type the handler receives can’t drift apart, because they share a single source. The handler is then just an ordinary Go function that takes those parameters and returns a result.

You register your tools with SetTools, call Chat, and that’s the whole of your involvement. The framework runs the ReAct loop and Chat returns the AI’s final text answer once the loop settles.

Two details that show it was built for real use

A couple of decisions in the loop tell you it’s meant for production, not a demo.

Tool errors don’t abort the conversation. When a handler returns an error, the framework doesn’t crash the loop. It hands the error back to the AI as a string, as just another observation. That’s deliberate, and it’s right. A real agent should be able to call a tool, watch it fail, and react: try different arguments, take a different route, or tell the user it couldn’t manage it. A loop that aborted on the first tool error would be far more brittle than the model driving it.

The loop is bounded. There’s a MaxSteps limit, default 20. An AI that gets confused could otherwise call tools forever, and a CLI command that never returns is a worse failure than a wrong answer. The cap guarantees the command terminates. The agent gets room to genuinely work a problem across many steps, but not infinite room to flail about in.

There’s also parallel tool execution: when the model asks for several tools in a single step (three independent file reads, say) the framework runs them concurrently rather than one after another, because there’s no reason to make the AI sit and wait out a sequence of things that don’t depend on each other.

Boiling it down

A text-only AI can describe your system; an AI that can call your functions can operate it. Bridging that gap means tool-calling, and tool-calling means the ReAct loop (reason, act, observe, repeat) whose orchestration is fiddly, identical every time, and not a problem worth solving twice.

go-tool-base’s chat package runs the loop for you. You define chat.Tool values (name, description, a tagged parameter struct that generates its own schema, a handler), call SetTools and Chat, and get the final answer. Tool errors go back to the AI as observations so it can recover, and a MaxSteps cap guarantees the command always terminates. You write Go functions. The framework turns them into things an agent can reach for.

Nobody reads the manual

Sun, 29 Mar 2026 00:00:00 +0000

Let me describe the actual lifecycle of a user meeting your CLI tool, because it’s a bit humbling. They run it. It doesn’t quite do what they expected. They run it again with --help. They get a wall of monospaced flag descriptions, skim it, don’t find the thing they wanted, and either give up or go and ask a human who already knows.

Your documentation might be magnificent. It doesn’t matter, because the user never reached it.

The manual loses on location, not quality

That’s the lifecycle, and notice exactly where it breaks. The documentation might be excellent. It might answer their precise question in full. It doesn’t matter, because it’s on a website, in another window, behind a search box, and the user is here, in the terminal, mid-task. The docs lost not on quality but on location. They simply weren’t where the work was.

go-tool-base’s answer starts with a decision about location: the documentation gets embedded into the binary itself. Your docs/ folder ships inside the tool, the same way its default config does. Wherever the tool is installed, the docs are right there alongside it, no network, no browser. That embedding is what makes everything else possible, and there are two things built on top of it.

A browser, in the terminal

The first is the docs command, and it’s not --help with extra steps. It launches a proper Terminal User Interface, built on Bubble Tea.

It has a sidebar, structured from the project’s own mkdocs.yml, so the docs are a navigable tree rather than one flat scroll. Markdown renders with real formatting through Glamour (colour, tables, lists, headings) instead of collapsing into monospaced soup. There’s live search across every page, regex included.

Compared with man and --help, the difference isn’t a nicer coat of paint. man gives you linear scrolling and grep; this gives you a structured tree, rich rendering and real search. It’s the documentation experience a modern developer expects, except it followed the tool into the terminal instead of demanding the user leave it.

A documentation assistant that won’t make things up

The second thing built on the embedded docs is the one I find genuinely transformative: docs ask.

The user doesn’t navigate anything. They just ask:

mytool docs ask "how do I point this at a self-hosted server?"

and get a direct, specific answer. Under the hood, the framework collates the tool’s embedded markdown and hands it to the configured AI provider (Claude, OpenAI, Gemini, Claude Local, any OpenAI-compatible endpoint) as the context for the question.

Now, “an AI answers questions about my tool” should immediately make you nervous, and the correct thing to be nervous about is hallucination. An AI that confidently invents a flag that doesn’t exist, or describes behaviour the tool simply doesn’t have, is worse than no assistant at all, because the user trusts it.

This is where embedding the docs pays off a second time, and it’s why I keep stressing that the corpus is closed. The model is instructed to answer only from the tool’s actual documentation, and the context it’s handed is exactly that documentation and nothing else. It isn’t drawing on a vague memory of similar tools from its training data. It’s answering from this tool’s real, shipped, version-matched docs. The corpus is small, closed and authoritative, which is the combination that keeps the answers honest. “Zero hallucination by design” isn’t a slogan about the model. It’s a property of bounding what the model is allowed to look at, which is the same instinct I leaned on with the mcp command: the safety comes from the boundary you drew, not from trusting the AI to behave itself.

There’s a nice second-order effect, too. The answer is always about the version of the tool the user actually has, because the docs were embedded into that build. No mismatch between a website documenting the latest release and the slightly older binary sitting on the user’s machine.

The upshot

Documentation usually loses to --help not on quality but on location: it’s in a browser, and the user is in the terminal. go-tool-base embeds the docs into the binary and surfaces them two ways: a docs command that’s a real TUI browser with a sidebar, rich markdown and search, and docs ask, which answers natural-language questions using the embedded docs as context.

Because that context is the tool’s own closed, shipped documentation and the model is told to use nothing else, the assistant stays grounded, and it’s always describing the exact version the user is holding. The fix for unread documentation was never to write more of it. It was to put it where the work happens and let it answer back.

BDD where it earns its place, and nowhere else

Sat, 28 Mar 2026 00:00:00 +0000

I have a slightly complicated relationship with BDD. I’ve watched it turn a tangled test suite into something the whole team could read and reason about, and I’ve watched it turn a perfectly good unit test into a paragraph of ceremonial English that nobody benefits from. So when go-tool-base brought in Cucumber-style BDD, the interesting decision wasn’t adopting it. It was being ruthless about where not to.

Two tests that hurt for different reasons

Most of go-tool-base’s tests are ordinary table-driven Go tests, and they’re absolutely fine. A function, a slice of input/expected pairs, a loop. Nobody needs Gherkin to understand a parser test.

But two areas were genuinely painful, and they were painful in the same way: the test had become harder to understand than the thing it was testing.

The first was pkg/controls, the service-lifecycle package. It runs a small state machine (Unknown, Running, Stopping, Stopped) with signal handling, health monitoring, restart policies and graceful shutdown all woven through it. The integration tests for graceful shutdown had grown to over three hundred lines of imperative goroutine and channel coordination. They worked. But reviewing them was a slog, and a test you can’t review with confidence is a test you can’t trust when it fails. The behaviour being checked, “when a shutdown signal arrives mid-startup, the controller stops cleanly”, was a simple sentence buried under a heap of synchronisation scaffolding.

The second was the CLI itself. init, update, doctor are user workflows. “Given a config file with a custom value, when I run init, then the custom value survives the merge.” That’s already a Given/When/Then; it just happened to be written out as Go.

Godog, and the line I drew

Godog is the official Go implementation of Cucumber. You write .feature files in plain Gherkin and bind each step to a Go function. The shutdown scenario stops being three hundred lines of channels and becomes this:

Scenario: graceful shutdown completes within the deadline
 Given a controller with two registered services
 When a shutdown signal is received
 Then both services stop in registration order
 And the controller reports a clean shutdown

The goroutine choreography doesn’t vanish, of course. It moves into the step definitions, written once and reused. What changes is that the scenario is now readable by someone who’s never opened the file before, including someone from an ops team who’ll never write a line of Go but absolutely has opinions about how shutdown should behave.

Here’s the part I want to dwell on, because it’s the part most BDD adoptions get wrong. The first design decision written down for this work was: strategic, not universal. Use Godog only where BDD adds clarity. Keep table-driven Go tests as the baseline everywhere else.

That sounds obvious written down. It is not obvious in practice, because BDD has a gravitational pull. Once a team has feature files, there’s a powerful urge to express everything as feature files, for consistency. And that’s how you end up with Gherkin scenarios for a pure function (Given the number 2, When I double it, Then I get 4) which is pure ceremony. You’ve wrapped a one-line table test in a paragraph of English and a step-definition indirection, and made it actively worse.

The test for whether BDD belongs is this: is this test a narrative, or is it a matrix?

A matrix is the same logic with many input/output pairs. That’s a table-driven test, that’s most unit tests, and Gherkin actively harms them. A narrative is a sequence of steps where the ordering and the state between steps is the thing under test, and that’s where Gherkin pays for itself. Lifecycle transitions are narratives. A user running three commands in sequence is a narrative. Doubling a number is not.

go-tool-base drew that line and stuck to it. Feature files live in features/ at the project root, where a non-Go developer can find and read them. Step definitions live in test/e2e/, kept well away from the unit tests. And the unit tests stayed exactly what they were, because they were already the right tool.

Made to fit, not bolted on

A couple of smaller decisions kept the BDD layer from feeling like a foreign object.

It runs under go test. There’s no separate Cucumber runner to install or remember. A godog.TestSuite is invoked from an ordinary TestFeatures(t *testing.T), so the BDD scenarios run in the same go test ./... as everything else. CI didn’t need a new concept bolted onto it.

And the CLI end-to-end tests build the gtb binary once and reuse it across every scenario. Compiling a binary per scenario would make the suite slow enough that people would quietly start skipping it, and a test suite people skip is just decoration. Build once, test many.

Stepping back

go-tool-base brought in Godog for BDD, but the decision worth writing about is the restraint. BDD was applied to exactly two things: the service-lifecycle state machine, where a 300-line goroutine tangle became a four-line scenario anyone can review, and CLI workflows, which are Given/When/Then by their very nature. Everywhere else, table-driven Go tests remained the baseline, because wrapping a matrix test in Gherkin makes it worse, not better.

The useful rule: BDD fits a narrative, ordered steps with meaningful state in between, and fights a matrix. Adopt it as a scalpel for the narratives. Resist the pull to turn it into a religion.

An AI interface that fits on one screen

Fri, 27 Mar 2026 00:00:00 +0000

The moment you decide a CLI tool should talk to an LLM, there’s a strong gravitational pull towards reaching for LangChain, or one of its many relatives. It’s the obvious move. It’s also, for most CLI work, a bit like hiring a removals firm to carry a single box up the stairs.

Let me explain why go-tool-base went the other way, and what “the other way” actually looks like.

The instinct, and why it overshoots

When you add AI to a tool, the instinct is to reach for the big general-purpose framework. LangChain and its relatives are capable, and they exist for a real need: orchestrating complex multi-step AI applications, with retrieval pipelines, memory stores, chains of calls, whole fleets of agents.

Now look at what a CLI tool actually needs from an LLM. It needs to send a prompt and get text back. Sometimes it wants structured data back instead of prose. Sometimes it wants to let the model call a few of the tool’s own functions. That’s pretty much the whole list.

Pulling in a framework built to orchestrate retrieval and agent swarms in order to do that is a poor trade. You take on a large new vocabulary of concepts, a wide dependency surface, and a great deal of abstraction you’ll never touch, all to perform three or four operations. The framework isn’t wrong. It’s just answering a far bigger question than the one a CLI tool is asking.

What go-tool-base chose instead

go-tool-base didn’t reach for a framework. The decision is on the record in its own design notes: before a single line was written, LangChain Go, go-openai, Vercel’s AI SDK and around ten other options were evaluated, and not one of them matched what a CLI framework actually needs. So the chat package was built deliberately small.

How small? The entire core ChatClient interface is four methods:

type ChatClient interface {
 Add(ctx context.Context, prompt string) error
 Chat(ctx context.Context, prompt string) (string, error)
 Ask(ctx context.Context, question string, target any) error
 SetTools(tools []Tool) error
}

Add appends a message to the conversation. Chat sends a prompt and returns text. Ask sends a prompt and returns a typed Go struct, the model’s answer unmarshalled straight into a value you defined. SetTools hands the model a set of your own functions it’s allowed to call. That’s the whole surface. Downstream code that uses AI never holds anything larger than this, and never has to know which provider is behind it.

The package’s own documentation has a word for this: right-sized. Large enough to solve genuine provider-abstraction complexity, small enough that the full interface fits on a single screen.

“Thin” is not the same as “does little”

This is the part worth being precise about, because “four methods” can sound like “barely does anything”, and that’s the wrong read entirely.

Behind those four methods sits genuinely awkward work. Five providers (OpenAI, Claude, Gemini, a locally installed claude binary, and any OpenAI-compatible endpoint) each with a different wire API, all normalised behind the one interface. A tool-calling loop. Structured output via JSON Schema, made to behave consistently across providers that each express it differently. Error normalisation. Token chunking.

The point of a thin abstraction is not that there’s little underneath it. It’s that the interface stays small while the implementation quietly absorbs the complexity. Four methods on the surface; five provider integrations and a tool-calling loop below the waterline. The thinness is a property of what the caller sees, not of what the package does. A reach-for-LangChain decision gets that backwards: it exposes the caller to all the machinery, whether or not the caller will ever need it.

The core stays small even as features grow

There’s a neat detail in how chat keeps the interface from creeping. The package also supports streaming responses and conversation persistence, both of which are real features with real surface area. Neither of them is in the four-method core.

Instead they’re separate, optional interfaces. A streaming-capable client also satisfies StreamingChatClient; a persistable one also satisfies PersistentChatClient. Code that wants those capabilities does a type assertion to ask for them, and code that doesn’t simply never sees them. So the common path stays four methods forever. New capabilities arrive as opt-in interfaces alongside the core, not as new methods bolted onto it. The thing that fits on one screen keeps fitting on one screen.

Extensible without forking, testable without a network

Two more properties keep the package small without making it limiting.

It’s extensible. The provider list isn’t closed. A RegisterProvider call lets any package contribute a new provider, and chat.New will route to it. You add a backend without forking pkg/chat or sending a patch upstream.

And it’s testable. The package ships generated mocks. A downstream tool’s AI features can be tested against a mock ChatClient returning canned responses, with no network, no API key, and no flakiness. Because the interface is four methods, that mock is trivial to set up and complete by construction. A sprawling framework interface is a sprawling thing to fake; a four-method one is not. (I’ll come back to testing AI code properly in a later post, because it deserves a whole article of its own.)

The right size

When a CLI tool needs AI, the instinct is a large framework like LangChain. For orchestrating retrieval pipelines and agent swarms, that’s exactly the right tool. For sending a prompt, getting a struct back, and letting the model call a few functions, it’s enormous overkill.

go-tool-base’s chat package is the deliberate alternative, chosen only after LangChain Go and a dozen others were weighed up and rejected. Its core ChatClient interface is four methods. Underneath sit five normalised providers, a tool-calling loop, structured output and error handling, but the caller sees four methods and never learns which provider is active. Streaming and persistence are opt-in interfaces beside the core, not additions to it. It extends without forking and tests without a network. Right-sized: the complexity is real, but it lives under the interface rather than in it.

The config key that quietly did nothing

Fri, 27 Mar 2026 00:00:00 +0000

I once spent the better part of an hour convinced a timeout setting was broken. I’d set it in the config file, the tool ignored it, and the code that read it looked perfectly correct. The setting was tiemout. I’d typed it wrong, and not one thing in the entire stack had thought that worth mentioning.

Config loaders are too polite

Most config loaders have the same agreeable flaw: they’ll read whatever’s in the file and quietly ignore anything they weren’t expecting. Put a key the tool doesn’t know about and it sails straight past. No error, no warning, nothing. The loader assumes you meant it, or assumes some other layer will care, and neither turns out to be true.

That politeness costs you in two directions. A key you misspelled is silently dropped, so the setting you thought you’d changed keeps running on its old value. And a key you forgot leaves the field at its zero value, which you then discover at runtime, usually at the least convenient moment, when something downstream divides by a timeout of zero. The file looked fine. It parsed fine. It was just quietly wrong, and nothing was watching for that.

The struct already knows the answer

The thing is, the program already has a complete description of what valid config looks like. It’s the struct you unmarshal into. The field names, the types, which ones matter. That description exists; it’s just not being used to check anything.

go-tool-base’s config package puts it to work. You hand it a tagged struct and it derives a schema from the tags, in pkg/config/schema.go:

// WithStructSchema derives a schema from a tagged Go struct.
// Supported tags: `config:"key" validate:"required" enum:"a,b,c" default:"value"`.
func WithStructSchema(v any) SchemaOption { ... }

So a feature’s config type carries its own rules inline:

type ServerConfig struct {
 Host string `config:"host" validate:"required"`
 Port int `config:"port" validate:"required"`
 LogMode string `config:"log_mode" enum:"text,json"`
}

There’s no second artefact to keep in sync, which is the same instinct go-tool-base leans on for structured AI output: the type is the schema, and the schema is a projection of the type, so the two can’t drift apart because there’s only one of them. Each package describes its own slice of config on its own struct, and NewSchema composes them into the schema the loaded config gets checked against.

Strict mode turns the typo into an error

Deriving the schema is half of it. The half that actually catches tiemout is this one, also from schema.go:

// WithStrictMode treats unknown keys as errors instead of warnings.
func WithStrictMode() SchemaOption { ... }

By default a key the schema doesn’t recognise is a warning: surfaced, but not fatal, which is the right call when a config file might legitimately carry extra keys for tools other than yours. Turn on strict mode and an unknown key becomes an error. tiemout isn’t in the schema, so the tool refuses to start and tells me which key it didn’t recognise, instead of shrugging and using the default for an hour while I lose my mind. The validator walks every key actually present in the file and checks it against the known set, so a typo has nowhere to hide.

What it deliberately doesn’t do

There’s one decision in here I think is worth calling out, because the obvious feature is conspicuously absent. The schema knows each field’s default value. It would be the easiest thing in the world to have validation fill in missing fields from those defaults.

It doesn’t, on purpose. Validation validates. It tells you what’s wrong and what to do about it, and it stops there. Defaults are a separate job, handled by the embedded default config that every feature ships and merges in before validation ever runs. Keeping the two apart means the validator has exactly one responsibility, and the defaults live in one place rather than being half in an embedded file and half injected by a check. A field’s default tag is there for the documentation and the error hint, not as a sneaky second source of values.

Errors you can act on

The output isn’t a bare boolean. Validation returns a result that separates the fatal from the advisory: the missing required field and the wrong type are errors that stop the tool; the unrecognised-but-harmless key is a warning that informs you without blocking. And because each problem carries the offending key by name and a hint about the fix, the message tells you what to change, in the spirit of errors that tell you what to do next.

The short version

A config loader that silently ignores keys it doesn’t recognise will, sooner or later, ignore one you meant. go-tool-base derives a validation schema straight from your tagged config struct, so there’s no separate schema to maintain, and strict mode promotes an unknown key from a quiet shrug to a real error that names the typo. It validates without injecting defaults, because defaults are the embedded config’s job and a validator with one responsibility is easier to trust. Set tiemout now and the tool tells you, which is roughly fifty-nine minutes sooner than I found out.

One variadic, and I'd already spent it

Thu, 26 Mar 2026 00:00:00 +0000

I had a constructor I was rather pleased with. Hand go-tool-base’s root command its props and as many sub-commands as you like, and off it goes. Then I needed to thread some config file paths through it, reached for the obvious “just add a parameter,” and discovered I’d already spent my one variadic with no second one going spare.

The ergonomics I’d happily bought

NewCmdRoot ends in subcommands ...*cobra.Command. That trailing ... is a small luxury: callers write NewCmdRoot(props, build, deploy, status) and never have to think about slices. Variadics are lovely for exactly this, the “and as many of these as you fancy” argument.

The parameter I couldn’t add

Then config arrived, and the root command needed to know about some extra configuration file paths. The instinct is to add a parameter. The instinct is wrong, because there’s nowhere legal to put it.

You can’t write NewCmdRoot(props, configPaths ...string, subcommands ...*cobra.Command). Go allows a function exactly one variadic, and it must be the final parameter. Two variadics results in a compile error before you’ve finished the line (assuming your IDE does compile time checks for you), and fairly so: at the call site, how would Go ever know where the strings stopped and the commands began? So the variadic I’d spent on sub-commands was spent. There wasn’t another to hand.

The choices, and the one I took

You can demote the variadic. Make it subcommands []*cobra.Command and you’re free to add configPaths []string next to it. Correct, and it breaks every existing call: NewCmdRoot(props, build, deploy) becomes NewCmdRoot(props, []string{}, []*cobra.Command{build, deploy}). Uglier at every site, to solve a problem only some callers have.

You can reach for functional options, and for plenty of go-tool-base’s constructors that is exactly what happened. But the root builder is the one everybody calls first, with the simplest signature in the codebase, and I didn’t want the common case lugging option machinery around for the sake of the rare one.

What I actually did was add a second door. From pkg/cmd/root/root.go:

// NewCmdRoot creates the root command with Props wiring and optional subcommands.
func NewCmdRoot(props *p.Props, subcommands ...*cobra.Command) *cobra.Command {
	return NewCmdRootWithConfig(props, []string{}, subcommands...)
}

func NewCmdRootWithConfig(props *p.Props, configPaths []string, subcommands ...*cobra.Command) *cobra.Command {
	// ...
}

The new argument goes in as a plain []string, sat before the variadic, which is perfectly legal: one variadic, still last. Callers who care about config use NewCmdRootWithConfig explicitly, and NewCmdRoot becomes a one-line wrapper that delegates with an empty slice, so every existing caller compiles untouched and none the wiser. Two doors into the same room, granted, but the original door is exactly where everyone left it.

What it comes down to

A trailing variadic is a slot you fill once. It buys gorgeous ergonomics for the “as many as you like” argument, and in exchange it quietly forecloses on ever appending another parameter, because the next one has nowhere to stand. Once it’s spent, new arguments come in as ordinary parameters before the variadic, and the kind thing to do for your callers is to put that behind a second constructor and let the original keep delegating.

So spend the variadic deliberately. Give it to the argument that genuinely wants to be a loose list, not the first one that happens to be plural, because you don’t get a second.

Half your users don't have eyes

Wed, 25 Mar 2026 00:00:00 +0000

Run a command in your favourite CLI tool and look at what comes back. Colour. Neatly aligned columns. A friendly little summary sentence. Lovely… if you happen to be a human with eyes.

But a good half of any tool’s users aren’t people at all. They’re scripts, CI pipelines, bits of automation. And that pretty output you’re so proud of is, to them, actively hostile.

Your tool has two audiences and only serves one

I made more or less this same point about AI assistants when I argued that your CLI is already an AI tool. The machines are users too. Here it isn’t an AI doing the calling, it’s a humble shell script, but the principle is identical.

Run a CLI command and look at what comes back. Colour. Aligned columns. A friendly summary sentence. It’s designed for a person reading a terminal, and for a person reading a terminal it’s great.

Now picture the other half of your users. A deploy script that needs to know which version is installed. A CI job that runs doctor and wants to fail the build on one specific check. A bit of automation gluing your tool to three others. None of them have eyes. They have parsers.

So what do they do with your beautiful human output? They butcher it. They grep for a keyword, awk out the third field, sed off a prefix. It works in the demo. Then someone rewords a status line, or adds a column, or the colour codes shift, and every script downstream breaks at once. Silently, too, because a broken grep returns nothing rather than an error. You changed a sentence and quietly took out somebody’s pipeline without ever knowing.

The human-readable output was never the contract. It just got used as one, because it was the only output there was.

Give the machines their own channel

The fix is not to make the human output more parseable. That’s a trap. You’d be constraining prose meant for people in order to satisfy programs, and end up serving neither of them well. The fix is to give programs their own output format, declared and stable, kept well away from the prose.

So every command built with go-tool-base gets a --output flag. Leave it alone and you get the friendly human rendering. Pass --output json and you get something a parser can actually rely on.

And not just some JSON. JSON with a fixed shape.

One envelope, every command

The temptation with JSON output is to let each command emit whatever structure happens to suit it. Don’t. A consumer scripting against five of your commands then has to learn five shapes, and “where’s the actual payload?” has a different answer every single time.

go-tool-base wraps every command’s JSON in one standard Response envelope:

{
 "status": "success",
 "command": "deploy",
 "data": {
 "environment": "production",
 "version": "1.4.0",
 "replicas": 3
 }
}

status says how it went. command says what produced it. data holds the command-specific payload, and only the payload. Every built-in command (version, doctor, update, init) emits exactly this shape. So does every command you write, because pkg/output hands you the envelope rather than letting you freelance:

format, _ := cmd.Flags().GetString("output")
w := output.NewWriter(os.Stdout, output.Format(format))

return w.Write(output.Response{
 Status: output.StatusSuccess,
 Command: "deploy",
 Data: result,
})

The consumer-side payoff is the whole point. A script can check .status without ever touching .data. It can pull .data.version and know the field is there because it’s typed, not scraped. It learns the envelope once, and every command in your tool, and every tool built on the framework, honours it. The contract is explicit, versioned, and the same everywhere, which is precisely what the abused human output never was.

The human output gets to relax

There’s a quiet second benefit, and it’s my favourite kind: the sort you get for free. Once programs have their own reliable channel, the human output is freed. It no longer has to stay accidentally parseable. You can reword a status line, add colour, restructure a table, make it genuinely nicer to read, and not break a single script, because no script is reading it any more. They’re all over on --output json, where the real contract lives.

Two audiences, two formats, each one actually suited to its reader. That’s the deal a CLI tool ought to be offering, and most of them don’t.

In short

A CLI tool that only emits human-readable output is only half-built, because half its users are programs that end up grep-ing prose and shattering the moment that prose changes. go-tool-base gives every command a --output json flag and one standard Response envelope (status, command, data) used identically by every built-in command and by anything you write through pkg/output. Machines get a stable, explicit, learn-it-once contract; humans get output that’s now free to be properly readable, because nothing fragile depends on its wording any more.

If your tool will ever be called by another program (and it will), give that program a front door. Don’t make it climb in through the window.

Lifecycle management for when your CLI grows up into a service

Tue, 24 Mar 2026 00:00:00 +0000

There’s a moment in the life of a lot of CLI tools where they stop being a CLI tool. Nobody quite decides it. It just happens. Someone needs the thing to also expose a little HTTP endpoint, or poll a queue, or run a scheduler, so it grows a serve command… and the honest command-line utility you wrote is suddenly a long-running service wearing a CLI as a hat.

And a service needs a whole pile of production plumbing that a one-shot command never did.

The command that stops being a command

go-tool-base is CLI-first. It is not CLI-only, and the reason is a pattern I’ve watched play out more times than I can count.

A tool starts its life as an honest command-line utility. It runs, it does its thing, it exits. Then someone needs it to expose a small HTTP endpoint. Or poll a queue. Or run a scheduler. So it grows a serve command, or a run command, and the moment it does, the thing that was a CLI tool is now a long-running service that happens to have a CLI bolted on the front.

And a long-running service needs a whole category of plumbing a one-shot command never did. It has to start things up in a sensible order. It has to shut them down gracefully when someone sends a SIGTERM, finishing in-flight work rather than dropping it on the floor. It has to tell an orchestrator whether it’s alive, and whether it’s ready. It has to do something sensible when one of its internal services quietly falls over at 3am.

Hand-rolled, that’s a few hundred lines of goroutine choreography, channel-wrangling and signal handling that every such tool reinvents, slightly differently and slightly wrong each time. It’s the first-afternoon problem all over again, just turning up later in the project’s life. So go-tool-base ships it: pkg/controls.

A controller and the things it controls

The model is small. A Controller manages any number of services. You register each with Register(id, opts...) and describe it with functional options: WithStart takes a StartFunc, WithStop a StopFunc. An HTTP server, a background worker, a scheduler, anything with a “begin” and an “end”.

You register your services with the controller and it owns their collective lifecycle. They share a common set of channels (errors, OS signals, health, control messages) so the whole set can react together. A SIGTERM doesn’t get caught by one service off in a corner; it reaches the controller, and the controller takes everything down in order, each StopFunc handed a context with a deadline so that one sulking service can’t wedge the whole shutdown forever.

That ordering and timeout handling is the bit nobody enjoys writing and everybody needs. Centralising it means a tool that adds a second service later inherits correct coordinated shutdown for free, rather than discovering on its first production SIGTERM that it only half shuts down.

Probes, because something is usually watching

If the service ends up in Kubernetes (and a lot of them do) the orchestrator wants to ask two different questions, and they really are different questions.

Liveness: are you alive, or are you wedged and in need of a kill? Readiness: are you alive and able to take traffic right now? A service can quite easily be live but not ready… still warming a cache, still waiting on a dependency. Conflate the two and you get yourself killed during a slow startup, or sent traffic before you can actually serve it.

controls keeps them separate. You attach a WithLiveness probe and a WithReadiness probe to a service, each just a function returning a health report, and the controller exposes them. The tool answers Kubernetes honestly, in Kubernetes’ own terms, without you hand-wiring two more HTTP handlers.

Self-healing, but only if you ask

The last piece is what happens when a service fails. A worker’s StartFunc returns an error. Health checks start failing. In a hand-rolled setup this is where you either crash the whole process or write yourself a bespoke restart loop.

controls has a supervisor that can restart a failed service for you, and the important word in that sentence is can. It’s off by default. A service is only supervised if you hand it a RestartPolicy at registration:

controls.WithRestartPolicy(controls.RestartPolicy{
 MaxRestarts: 5,
 InitialBackoff: time.Second,
 MaxBackoff: 30 * time.Second,
 HealthFailureThreshold: 3,
})

With a policy in place, the controller restarts the service if its StartFunc errors out, or if it racks up more consecutive health-check failures than the threshold allows. Restarts back off exponentially, from InitialBackoff up to a MaxBackoff ceiling, so a service that’s failing because its database is down doesn’t sit there hammering that database flat with a tight restart loop. MaxRestarts caps the attempts, because a service that’s failed five times in a row is not going to be rescued by a sixth go, and at that point honest failure beats a thrashing pretence of health.

Opt-in matters here. Automatic restarts are exactly right for a resilient daemon and exactly wrong for a tool where a failure should stop the line and get a human’s attention. The framework doesn’t make that call for you. It gives you the supervisor and lets you point it at the services that genuinely want it.

The bottom line

A surprising number of CLI tools become long-running services the day they grow a serve command, and the day they do, they need coordinated startup, graceful ordered shutdown, real liveness and readiness probes, and a considered answer to a service falling over. That’s a few hundred lines of fiddly, easy-to-get-wrong plumbing.

pkg/controls provides it: a Controller over Controllable services with shared channels and deadline-bounded graceful shutdown, separate Kubernetes-style liveness and readiness probes, and an opt-in supervisor that restarts failed services with exponential backoff and a restart ceiling. Your tool can start as a command and grow into a daemon without that growth turning into a rewrite.

CLI-first, but not stuck there.

Middleware for CLI commands, not just web servers

Tue, 24 Mar 2026 00:00:00 +0000

Every CLI tool past a certain size grows a category of logic that doesn’t really belong to any one command, and yet has to happen for loads of them. Timing. An auth check. Panic recovery, so a crash becomes a clean error instead of a stack-trace all over someone’s terminal. A log line saying the command started and how it finished.

Web frameworks sorted this out years ago. CLIs, for some reason, mostly still copy-paste it around.

The logic that belongs to no single command

That category of logic doesn’t belong to any one command, yet needs to happen for many of them. Time how long the command took. Check the user is authenticated before a command that needs it. Recover from a panic so a crash becomes a clean error rather than a stack-trace vomited across the screen. Log that the command started and how it ended.

None of that is the command’s job. The deploy command’s job is to deploy. But timing and recovery and auth still have to happen around it, and around build, and around sync.

Put that logic inside each command’s RunE and you’ve copied the same six lines into thirty functions, which means thirty places to fix when the logging format changes and thirty chances to forget one of them. Cross-cutting concerns copied by hand don’t stay consistent. They drift, every time.

Web frameworks already solved this

This is not a new problem. It’s about the oldest problem in web frameworks, and they settled on an answer a long time ago: middleware. Gin has it, Echo has it, every HTTP stack you’ve ever touched has it. A middleware is a wrapper that sits around a handler, runs its cross-cutting logic, and calls through to the handler in the middle.

A CLI command is, structurally, just a handler too. So go-tool-base brings the same pattern to the Cobra command tree, with the same functional Chain shape:

type Middleware func(
 next func(cmd *cobra.Command, args []string) error,
) func(cmd *cobra.Command, args []string) error

A middleware receives the next handler in the chain and returns a new handler that wraps it. You compose a stack of them, and each command’s real RunE runs in the middle of the onion. Write the timing logic once, as one middleware, and every command in the chain is timed. Change the log format once and all thirty commands change with it, because there was only ever one copy. (The “write it once, in a place where everyone inherits it” drum again, which I will keep banging until the series runs out.)

“But Cobra already has PreRun”

It does, and this is the objection worth answering properly, because Cobra ships PersistentPreRun and PreRun hooks and they look, at a glance, like they cover this.

They don’t, and the reason is structural. A PreRun hook is a thing that happens before the command. That’s all it is. It can’t run anything after. It can’t wrap the command in a defer. It can’t catch a panic the command throws. It can’t measure how long the command took, because measuring a duration needs a start point and an end point, and the hook only owns the start.

A middleware wraps the entire execution. Because it’s a function that calls next() in its own body, it straddles the command (with the handler signature abbreviated to HandlerFunc here for readability):

func TimingMiddleware(next HandlerFunc) HandlerFunc {
 return func(cmd *cobra.Command, args []string) error {
 start := time.Now()
 err := next(cmd, args) // the command runs here
 log.Debug("command finished", "took", time.Since(start))
 return err
 }
}

Before, after, and around. A recovery middleware can put a defer recover() in place that a PreRun hook structurally cannot. An auth middleware can check a condition and return an error instead of calling next() at all, refusing to let the command run in the first place. PreRun can’t veto the command; it runs, and then the command runs regardless.

PreRun is a notification that the command is about to happen. Middleware is control over whether and how it happens. For genuine cross-cutting concerns you need the second thing, not the first.

To sum up

Timing, auth, recovery and logging are cross-cutting concerns: necessary for many commands, owned by none. Hand-copied into every RunE, they drift out of sync. Web frameworks fixed this with middleware years ago, and a CLI command is structurally just another handler.

go-tool-base brings the functional Chain middleware pattern to the Cobra command tree. A middleware wraps a command’s whole execution, so it acts before and after and can decide whether the command runs at all… strictly more than Cobra’s PreRun hooks, which only fire beforehand and can’t wrap, recover, time, or veto. Write the concern once, wrap the chain, and every command inherits it consistently.

A logging interface that doesn't leak its backend

Mon, 23 Mar 2026 00:00:00 +0000

The same tool, in two different lives, wants two completely different kinds of log.

On my laptop I want logs I can actually read: colour, alignment, friendly timestamps. The very same tool running as a daemon in a container wants none of that. It wants structured JSON, one object a line, ready for a log aggregator to swallow. And in a test I want the logger to shut up entirely. The interesting question is what it costs you to move between the three.

The same tool wants different logs

On a developer’s machine the tool is a CLI. You want logs that are pleasant to read in a terminal: colour, alignment, human-friendly timestamps. The charmbracelet logger does that beautifully.

Then the very same tool grows a serve command and gets deployed as a daemon in a container. Now coloured terminal output is worse than useless. The log aggregator wants structured JSON, one object per line, machine-parseable. slog does that.

And in tests you want neither. You want the logger to exist, satisfy the interface, and stay completely silent.

That’s three different logging backends, wanted by one tool across three different lives. So what does switching between them actually cost?

What it costs depends on what your packages imported

If your packages import a concrete logger, if pkg/config and pkg/setup and twenty others each have import "github.com/charmbracelet/log" and take a *log.Logger, then the backend is welded into the entire codebase. Switching to JSON for the container build means editing the import and the parameter type in every single one of those packages. The backend has leaked. A detail that should have been one decision has become a property of a hundred files.

go-tool-base doesn’t let it leak. Every package in the framework accepts a logger.Logger, an interface, and nothing else. No package anywhere imports a concrete logging library. A package states, in its types, “I need something I can log through”, and stops right there. It has no idea, and no way to find out, what’s actually on the other end.

// what every package depends on
type Logger interface {
 Debug(msg string, keyvals ...any)
 Info(msg string, keyvals ...any)
 Warn(msg string, keyvals ...any)
 Error(msg string, keyvals ...any)
 // ...
}

The backend gets chosen once, at the top, when the tool builds its Props. It travels down to every package as the interface, through the Props container. The packages underneath never see the concrete type, so the concrete type can change without a single one of them noticing. (There’s that “decide it once, in one place” theme again. I did warn you it runs through everything.)

Three backends, and the swap is one line

go-tool-base ships three implementations of that interface:

charmbracelet (logger.NewCharm(w, opts...)). Coloured, styled, for humans at a terminal. The CLI default.
slog JSON, a slog-backed backend emitting structured JSON, for daemons and containers feeding a log aggregator.
noop, which does precisely nothing, for tests that want a real Logger and total silence.

Switching the tool from a friendly CLI logger to container-ready JSON is a change to the one line in main() that constructs the logger. That’s the lot. pkg/config doesn’t change. pkg/setup doesn’t change. None of the twenty packages change, because none of them ever knew which backend they had. The decision was always one line; the interface is what kept it one line.

The noop backend deserves its own mention, because it’s the one people underrate. A test for a command shouldn’t be spraying log output all over the test run, but the command still needs a non-nil Logger to function. logger.NewNoop() gives you exactly that: interface satisfied, output binned, test quiet. And because it’s just another implementation of the same interface, no test needs any special logging machinery. It passes a different backend, exactly the way the container build does.

The general shape

There’s nothing exotic going on here. It’s “depend on interfaces, not implementations”, which every Go developer has had drilled into them at some point. The bit worth holding onto is where the rule actually pays out, and it’s at the seams between a stable core and a detail you know full well you’ll want to vary.

A logging backend is exactly such a detail. You will want it different in a terminal, in a container, and in a test. So the thing your code depends on has to be the interface, and the concrete backend has to be chosen at one well-known point and nowhere else. Get that boundary right and “we need JSON logs in production” is a one-line change. Get it wrong and it’s a refactor and a bad afternoon.

What it comes down to

One tool legitimately wants three different logging backends across its life: coloured output in a terminal, structured JSON in a container, silence in a test. The cost of moving between them is decided entirely by whether your packages imported a concrete logger or an interface.

go-tool-base’s packages depend only on logger.Logger, never a backend. Three implementations ship (charmbracelet, slog JSON, noop) and the backend is chosen once, in main(), then carried everywhere as the interface through Props. Switching is one line at the top, because the detail was never allowed to leak into the hundred files below it.

Errors that tell the user what to do next

Sun, 22 Mar 2026 00:00:00 +0000

Here’s an error message I’ve been on the receiving end of more times than I’d care to count:

error: failed to read config file

True. Also completely useless! I now know something is broken and I haven’t the faintest idea what to do about it. Which file? Why couldn’t it be read? Should I create it, run some init command, fix a permission, set an environment variable? The message states the problem and then abandons me at it, rather like a sat-nav cheerfully announcing “you have arrived” in the middle of a motorway.

A message is not a fix

The instinct, the moment you notice this, is to go and write a better message:

error: failed to read config file at ~/.config/mytool/config.yaml.
Run 'mytool init' to create one, or set MYTOOL_CONFIG to point at an existing file.

Better for the human, no question. But look at what you’ve just done to the error as a value. The recovery advice is now welded into the error string. Any code that wants to ask “is this the config-missing error?” is reduced to substring-matching English prose. Reword the advice and you break the check. So you’ve helped the user and quietly sabotaged the program at the same time, because you’ve made one poor little string do two completely incompatible jobs… being a stable identity for code, and being friendly guidance for people.

Why I changed error libraries

go-tool-base started out on github.com/go-errors/errors. It’s a perfectly fine library and it gave us stack traces. What it didn’t give us was any way to attach human guidance to an error without shoving it into the message string. So the codebase did exactly the daft thing I just described: multi-line suggestion text baked straight into errors.Errorf calls, user-facing content and programmatic identity all mashed into one value.

That’s the whole reason for the migration to github.com/cockroachdb/errors. Not novelty, and not because I fancied a weekend of find-and-replace. One specific capability: cockroachdb/errors lets you attach a hint to an error as a separate, structured field.

return errors.WithHint(
 errors.New("failed to read config file"),
 "Run 'mytool init' to create one, or set MYTOOL_CONFIG to point at an existing file.",
)

Now there are two things, cleanly apart. errors.New("failed to read config file") is the identity… stable, matchable, the program’s handle on the error. The hint is the guidance… for the human, and rewordable as much as you like without breaking a single check, because no check ever looks at it. errors.Is and errors.As work properly through every wrapper layer, so code matches on identity and never has to read prose.

The migration brought a few other things worth having. Stack traces print with a plain %+v instead of a type assertion. Errors can carry structured, machine-readable metadata. Multiple errors from concurrent work can be combined as a first-class value. But the hint is the one that actually changed the user’s day, because the hint is the recovery step, stored where it belongs.

One door out, and it knows where the help is

Separating the hint is only half of it. The other half is making sure those hints actually reach the user, every time, and that comes down to having a single way out.

Every go-tool-base command returns its errors the idiomatic Cobra way, through RunE. They all funnel into one Execute() wrapper at the root, which routes every error (runtime failure, flag parse error, pre-run failure) through one ErrorHandler. One door out. So error presentation gets decided in exactly one place, and no command can render an error differently from the command sat next to it.

And because there’s one handler, it can pull off something the individual commands never could. The framework knows your tool’s metadata, including its configured support channel, be it a Slack workspace or a Teams channel. So the error handler can finish a fatal error not just with the what and the recovery hint, but with where to go if the hint didn’t help:

error: failed to read config file
hint: Run 'mytool init' to create one, or set MYTOOL_CONFIG.
 Still stuck? Ask in #mytool-support on Slack.

The user is never left at a dead end. The error tells them what broke, the hint tells them the most likely fix, and if that’s still not enough the handler tells them which door to go and knock on. A failure becomes a signpost instead of a full stop.

The short version

An error that only reports what went wrong leaves the user stranded, and the obvious fix (writing the recovery advice into the message) quietly wrecks the error as a value, because now your code has to substring-match prose just to work out what it’s looking at.

go-tool-base moved from go-errors to cockroachdb/errors to get hints: a structured, separate field for human guidance that leaves the error’s identity clean for errors.Is and errors.As. Every command’s errors leave through one Execute() wrapper and one ErrorHandler, so presentation stays consistent, and because that handler knows the tool’s support channel it can point a stuck user at real help.

State the problem for the program. Give the fix to the human. And for pity’s sake, keep the two in different fields.

Many embedded filesystems, one merged view

Sat, 21 Mar 2026 00:00:00 +0000

Go’s embed package is one of those features that makes you slightly giddy the first time you use it. One //go:embed directive and your default config, your templates, your docs are all baked into the binary. The tool just works the moment it’s installed, with nothing external to lose or forget to ship.

And then you go and build something modular on top of it, and you discover the catch nobody warned you about.

`embed.FS` is an island

An embed.FS has a property that’s easy to miss until it bites: it’s local to the package that declared it. The //go:embed directive can only see files at or below its own source file. So in any project bigger than a toy, you don’t have an embedded filesystem. You have many. The root package embeds one. Each feature, each subcommand that ships its own templates or defaults, embeds another. They’re islands, one per package, and Go gives you no native way to make them behave as a whole.

For most files that’s perfectly fine. A feature’s templates can stay on the feature’s island; nothing else needs them.

It stops being fine the moment features need to contribute to something shared.

The shared-config problem

Here’s the case that forces the issue. A go-tool-base tool has a global config.yaml of defaults, embedded at the root. Now you add a feature, and that feature has its own configuration keys, with their own sensible defaults.

Where do those defaults go?

The naive answer is: edit the root config.yaml and add the feature’s section. And that’s a genuinely bad answer, because it inverts the dependency. The root config now has to know about every feature. Add a feature, edit the centre. Remove one, edit the centre again. The central file becomes a pinch point that every feature has to reach into, and a modular architecture where you can’t add a module without editing the core isn’t really modular at all… it just has more files.

What you actually want is for the feature to ship its own slice of default config, on its own island, and for the global config the tool reads to somehow already contain it. The feature contributes; the centre doesn’t budge.

`props.Assets`: merge the islands

That’s the job of props.Assets. (Yes, it lives on Props, the load-bearing container I keep going on about. Most of the good stuff does.) It’s a layer that implements the standard fs.FS interface, and into it you Register each embed.FS under a name:

// root main.go
Assets: props.NewAssets(props.AssetMap{"root": &assets}),

// a feature's command constructor
//go:embed assets/*
var assets embed.FS

func NewCmdFeature(p *props.Props) *cobra.Command {
 p.Assets.Register("feature", &assets)
 // ...
}

Now Props carries one Assets value that represents all the islands as a single filesystem. The root’s files and every registered feature’s files, addressable through one fs.FS. Each registration is named, so the islands stay individually identifiable, but they read as one.

That alone solves the addressing problem. The genuinely clever part is what happens for structured files.

Opening a file that exists in several places

When you Open a path through props.Assets and that path has a structured extension it recognises (.yaml, .yml, .json, .toml, .csv, and a few more) it doesn’t simply return the first match it stumbles across. It does this:

Discovery. It finds every instance of that path, across every registered filesystem.
Parsing. It unmarshals each one.
Merging. It deep-merges the parsed data, using mergo.
Re-serialisation. It hands you back a single fs.File whose contents are the combined, merged result.

So picture the shared-config problem again, only solved this time. The root ships a config.yaml with the base defaults. Each feature ships a config.yaml on its own island carrying only its own keys. Nobody edits anybody else’s file. When the init command opens config.yaml through props.Assets, it doesn’t get the root’s copy. It gets the deep-merge of the root’s copy and every registered feature’s copy: one config.yaml that contains every default in the tool, assembled at runtime from contributions that never knew about each other.

A feature contributes its defaults simply by existing and registering. The centre never changes. That’s the modular property the naive approach couldn’t give you, and it generalises well beyond config… the same merge applies to a shared commands.csv, or any structured file features want to add rows or keys to.

There’s also a Mount method for attaching an arbitrary fs.FS at a virtual path, which is handy for surfacing something external (a temp directory, say) as part of the same tree. But the structured merge is the feature that really earns Assets its place.

Boiling it down

embed.FS is per-package by design, so a modular CLI ends up with many embedded filesystems, one island per feature. Most of the time that’s fine. It fails specifically when features need to contribute to a shared resource like the global config.yaml, because the naive fix forces every feature to reach in and edit a central file.

props.Assets merges all the registered islands into a single fs.FS, and for structured files it goes further: opening a .yaml, .json or .csv discovers every copy across every island, deep-merges them, and returns the combined whole. A feature drops its own defaults onto its own island, registers, and the merged config the tool reads already includes them. Contribution without coupling, which is rather the whole point of being modular in the first place.

Props: the container that does the heavy lifting

Sat, 21 Mar 2026 00:00:00 +0000

I name-dropped Props back in the introduction and then rather glossed over it, which was a bit unfair of me, because it’s the single most important design decision in the whole framework. So let’s give it the attention it actually deserves.

And the best place to start, oddly enough, is the name.

Start with the name

The container at the centre of go-tool-base is called Props, and the name is doing real work, so we’ll start there.

It is not short for “properties”, though it does hold a few. A prop is the heavy timber or steel beam that stops a structure quietly collapsing in on itself. And for anyone who follows the rugby: a prop is the position in the scrum, the broad-shouldered forward whose entire job is to provide structural support so everyone else can get on with the game.

That’s the design brief, in a single word. Props is not where the clever, flashy work happens. It scores no tries. It’s the thankless, load-bearing thing that holds the framework up so that your actual command logic gets to be the interesting part. Understand the name and you understand what the struct is for.

What it carries

Props is the single object passed to every command constructor in a go-tool-base tool. It holds the dependencies a command might need:

Tool, metadata about the CLI (name, summary, release source).
Logger, the logging abstraction.
Config, the loaded configuration container.
FS, a filesystem abstraction (afero), so a command never touches the real disk directly.
Assets, the embedded-resource manager.
Version, build information.
ErrorHandler, the centralised error reporter.
Collector, the telemetry collector (always present, a no-op when telemetry is off).

A command constructor’s signature is, accordingly, boring on purpose:

func NewCmdExample(p *props.Props) *cobra.Command { ... }

One parameter. Everything the command could possibly need is reachable through it. No globals, no init()-time wiring, no twelve-argument constructor that quietly grows a thirteenth argument next month.

Why a struct, and not `context.Context`

Here’s the design decision I actually want to defend, because it’s the one Go developers tend to raise an eyebrow at. Go already has a well-known way to carry things through a call tree: context.Context. So why not just put the logger and the config in the context and pass that around?

Because context.Context carries its values as interface{}, and that’s the wrong trade for dependencies.

Pull a dependency out of a context and you get this:

l := ctx.Value("logger").(logger.Logger) // a runtime type assertion

That one line has two separate ways to hurt you. The key is a bare string, so a typo compiles perfectly happily and then fails at runtime. The type assertion is unchecked, so if the wrong thing is sitting under that key, your tool panics in front of a user. Neither failure is visible to the compiler. Neither is visible to your IDE. You find out when it breaks, which is to say at the worst possible time.

Pull the same dependency out of Props and you get this:

p.Logger.Info("starting") // a field access

p.Logger is a typed field. If it doesn’t exist, or you’ve used it wrong, the code simply doesn’t compile. Your IDE autocompletes it. Refactor the Logger interface and every misuse lights up at build time. There’s no runtime type assertion, because there’s no interface{} to assert from in the first place.

context.Context is the right tool for what it was designed for: cancellation, deadlines, request-scoped signals that genuinely cross API boundaries. It’s the wrong tool for “here are my program’s services”, because it trades away the compiler’s help for a flexibility you really don’t want here. Dependencies should be declared, somewhere the compiler checks them. Props is that somewhere.

What you get back for it

That one decision pays out in three currencies.

Testability. A command is now a pure function of its Props. To test it, you build a Props with the doubles you want (an in-memory FS instead of the real disk, a no-op Logger, a config you’ve populated by hand) and call the constructor. No global state to reset between tests, no monkey-patching, no init() order to puzzle over. The dependency is an argument, so the test just passes a different one.

Consistency. Cross-cutting changes have exactly one place to happen. When the global --debug flag flips the log level, it does so on the Logger inside Props, and because every command reads its logger from the same Props, every command gets the new level. No command can drift, because none of them owns its own copy.

Extensibility. Adding a new framework-wide service is just adding a field to one struct. Every command can immediately reach it; none of them needed touching to make it reachable.

To sum up

Props is the dependency-injection container at the heart of go-tool-base: one struct, passed to every command, holding the logger, config, filesystem, assets, error handler and tool metadata. It’s a concrete struct rather than a context.Context payload entirely on purpose, because dependencies belong somewhere the compiler can check them, not behind a string key and a hopeful runtime type assertion. That single choice buys you testability, consistency and easy extension.

The name says it best, really. Props doesn’t score the tries. It’s the broad-shouldered thing in the scrum that stops the whole framework folding, so the rest of your code is free to go and play.

Design your whole CLI in one file

Fri, 20 Mar 2026 00:00:00 +0000

Here’s a question that sounds trivial and really isn’t: where, exactly, does a CLI tool’s structure live? Not the logic of each command… the structure. Which commands exist, what they’re called, which flags they take, what’s nested under what.

I’d never properly thought to ask it until go-tool-base forced me to, and the answer turned out to be a little bit embarrassing.

Where does a CLI’s structure actually live?

Picture a CLI tool with twenty commands, some nested under others. In a typical project, where does its structure live? The answer is “smeared across the codebase”. It’s in twenty cmd.go files. It’s in the AddCommand calls that stitch them together. It’s in the flag registrations. To understand the shape of the tool you have to read all of it and assemble the picture in your head, because the picture exists nowhere as a single thing you can point at.

That’s a strange state of affairs for the single most important design fact about a CLI. The command tree is the tool’s interface, it’s the thing users actually touch, and yet it hasn’t got a home.

The manifest gives it one

go-tool-base’s generator gives that structure a home: .gtb/manifest.yaml. The manifest is a single readable file describing the command tree. Every command, its name, its short description, its flags, its place in the hierarchy, whether it carries assets or an initialiser. The shape of the whole tool, in one place you can open and read top to bottom.

And the manifest isn’t documentation about the project. It’s the thing the project’s wiring is generated from. When you run regenerate project, the generator reads the manifest and rebuilds the boilerplate to match it: the command registration, the AddCommand wiring, the flag definitions. The manifest is the source of truth, and the Go wiring is its output.

Design-first, when you want it

This unlocks a way of working that the smeared-across-the-codebase approach simply can’t offer. You can design the interface first, in the manifest, and let the code follow.

Want to rename a command? Edit one line in the manifest, run regenerate, and the rename propagates through every wiring file that ever mentioned it. Want to move a subcommand under a different parent? Change its place in the manifest hierarchy and regenerate. Want to add a flag to three related commands? Add it in the manifest, in three obvious places, and regenerate, instead of going on a little hunting expedition for three flag-registration blocks scattered across the tree.

You’re editing the tool’s interface as a design, in the file whose entire job is to hold that design, and the generator does the mechanical donkey-work of making the code reflect it. The thing you change is the thing that describes the structure. The code is downstream.

If that shape sounds familiar, it should. It’s the same instinct behind spec-driven and test-driven development: write down what the thing should be before you assemble how it works, and keep that statement of intent as a first-class, living artefact rather than a comment that quietly rots in a corner. The manifest is a spec for your command tree, and regenerate is what keeps the implementation honest to it.

It doesn’t trap you

There’s an obvious worry about any generated-from-a-manifest system: am I now locked into editing the manifest? What if I just want to open a Go file and write some Go like a normal person?

You can. The generator is careful not to own everything. It owns the wiring (the registration and the structural boilerplate) and it leaves your command logic well alone. The RunE function where your command actually does its work is yours; the manifest hasn’t got an opinion about it. And the generator tracks the files it produces by content hash, so if you do hand-edit something it generated, regeneration notices and asks before overwriting rather than steamrolling you. That mechanism turned out interesting enough to get its own post.

So the manifest is an option, not a cage. Design-first via the manifest when that suits the change. Drop into Go directly when that suits it better. The two stay in sync because regeneration reconciles them, not because one of them has been forbidden.

Pulling it together

A CLI’s command tree is its most important design surface, and in most projects it has no single home… it gets reconstructed in your head from twenty scattered files every time you need to reason about it. go-tool-base gives it one: .gtb/manifest.yaml, a readable description of the whole tree that the generator rebuilds the wiring code from. Edit the manifest, run regenerate, and the boilerplate follows.

It makes CLI structure something you design in one place, in the spirit of spec-driven development, while still leaving you free to write Go directly when that’s the better tool for the job. The manifest is the spec for your interface. The generator just keeps the code faithful to it.

Scaffolding that respects your edits

Fri, 20 Mar 2026 00:00:00 +0000

When I introduced go-tool-base I made a passing promise to come back to “the generator that won’t clobber your edits”. This is me keeping it, partly because it’s the feature I’m quietly most proud of, and partly because it took the most head-scratching of anything to get right.

The problem it solves is one that every code generator runs into eventually, usually the hard way and usually at the worst possible moment.

The generator’s awkward second act

A project generator has an easy first act. gtb generate project, and you’ve got a complete, wired, idiomatic Go CLI project. Everyone’s happy, me included.

The second act is the hard one. The framework moves on. A convention changes, a new built-in capability appears, the recommended CI shape shifts. Your project, scaffolded three months ago, is now subtly out of date, and you’d quite like the generator to drag it back up to spec.

Except by now it isn’t a fresh scaffold. It’s your project. You tuned the CI workflow. You rewrote the justfile. You added a stanza to the Dockerfile that took an afternoon and a fair bit of swearing to get right. The generated files and your edited files are one and the same files.

A naive generator handles this with breathtaking confidence: it regenerates everything from the template and overwrites the lot. Run it once, lose your afternoon. You learn that lesson exactly once and then never run regeneration again, which means the upkeep feature you were sold is dead on arrival. A scaffold you can’t safely re-run is just a one-shot cp with extra steps.

What the generator needs to know

The thing standing between “safe to overwrite” and “absolutely do not” is a single fact: has this file changed since the generator last wrote it?

If it hasn’t, the file is still pristine boilerplate and the generator owns it. Overwrite away. If it has, a human has been in there, and the generator must not touch it without asking first.

The generator can’t just eyeball that, of course. It needs a record. So every time gtb generate writes a file, it computes a SHA-256 of the content and stores it in the project’s manifest, .gtb/manifest.yaml, as a Hashes map of relative path to hash. The manifest is the generator’s memory of the exact bytes it last produced.

Regeneration becomes a three-way decision

With that record in hand, regeneration stops being “overwrite everything” and becomes a per-file decision with three branches.

The file doesn’t exist. Easy. Write it, store its hash.

The file exists and its current hash matches the manifest. It’s byte-for-byte what the generator last wrote, so nobody has touched it. The generator owns it outright, regenerates from the template and updates the stored hash. No prompt, no fuss. This is the common case, and it’s silent precisely because it’s safe.

The file exists and its hash does not match. Someone has been in there since generation. The generator stops and asks. It will not silently overwrite your hard-won afternoon. You decide: take the new version, or keep yours.

The detail I’m genuinely fond of is what happens when you decline. Declining is non-fatal. Generation carries on with the rest of the files, and the manifest keeps the file’s stored hash rather than dropping it. That matters more than it looks, because it means the file stays tracked. Next time you regenerate, the generator can still tell that file has been modified, and still asks. Skipping a file once doesn’t quietly evict it from the generator’s awareness forever. It stays a known, watched, customised file across every future run.

When you want it to stop asking

Per-file prompting is the right default, but for files you’ve permanently taken ownership of, being asked on every single regeneration is just noise. If you’ve rewritten the CI workflows wholesale and you are never, ever going back to the generated version, you don’t want a prompt. You want the generator to leave them well alone and not bring it up again.

That’s what .gtb/ignore is for. It sits next to the manifest and takes gitignore-style patterns:

# I own the CI workflows now
.github/workflows/**

# ...except the release workflow, keep that managed
!.github/workflows/release.yml

# and my build config
justfile
Dockerfile

Anything matching is skipped during regeneration with no prompt at all. Patterns evaluate top to bottom and later ones win, so the negation (!) behaves the way you’d expect from .gitignore: exclude a whole directory, then claw one file back.

It’s a deliberate escalation ladder. Unmodified files are handled silently. Modified files get a prompt. Files you’ve formally claimed get total silence. Each rung asks for less of your attention than the last, and you choose how far up to climb, file by file.

Stepping back

A generator earns its keep twice: once when it scaffolds your project, and then continuously, every time it drags that project back up to the framework’s current shape. The second job is worth nothing if regeneration flattens your customisations, because you’ll simply stop running it, and who could blame you.

go-tool-base’s generator gets around that by remembering. It hashes every file it writes into .gtb/manifest.yaml, and on regeneration it re-hashes before overwriting: unchanged files it owns and updates silently, changed files it stops and asks about, and .gtb/ignore lets you mark files as permanently yours. Skipped files stay tracked, so the generator never loses sight of what you’ve made your own.

The point of a scaffold isn’t the first five minutes. It’s that you can still run it in month three without holding your breath.

Your CLI is already an AI tool

Thu, 19 Mar 2026 00:00:00 +0000

“Make it work with AI” has become one of those requests that lands on a developer’s desk with a thud and not much further detail attached. My instinct, the first time, was to brace for a big lump of integration work… a bespoke adapter for this assistant, another for that one, a treadmill of little wrappers stretching off into the distance.

Turns out I’d already done most of the work. So have you, if your CLI tool is any good. Let me explain what I mean.

You already described your capabilities

Stop and think for a second about what a well-built CLI tool actually is. It’s a set of named operations, each with a human-readable description, each taking a set of typed, named, documented parameters. You wrote all of that already, because a CLI without it is unusable by people.

Now look at what an AI assistant needs in order to call a tool. A set of named operations. A description of each, so it knows when to reach for them. A typed parameter schema for each, so it knows how to call them.

It’s the same list! A good CLI is already, structurally, a description of a set of capabilities. The information an AI agent needs isn’t extra work you have to go and do. It’s work you finished the moment your --help output was any good.

The only thing missing is a translator. Something that takes “this is a CLI” and presents it as “this is a set of tools an AI can call”.

MCP is that translator, and it’s a standard

The temptation, when you want your tool to be AI-usable, is to sit down and write an integration. A little adapter for Claude Desktop. Another for Cursor. Another for whatever turns up next month. Each one a bespoke wrapper, each one a thing to maintain, and the list never stops growing because new assistants keep appearing. That’s the treadmill I was bracing for.

The Model Context Protocol exists to kill that list. MCP is an open standard for how an AI model discovers and calls local tools. Implement it once and your tool works with every assistant that speaks it. Write once, not once-per-client.

So go-tool-base implements it once, in the framework, for everyone. (That’s rather the theme of this whole series, if you hadn’t spotted it yet… do the annoying thing once, properly, in a place where every tool inherits it.)

The `mcp` command, and the mapping it does for free

Every tool built on go-tool-base inherits a built-in mcp command. Run it:

mytool mcp

and the tool starts a JSON-RPC server over standard I/O, speaking MCP. That’s the whole user-facing surface. One command.

Behind it, the framework walks your Cobra command tree and maps it straight onto MCP tool definitions:

Each command becomes a tool.
Each command’s short description becomes the tool’s description, the text the AI reads to decide whether this is the tool it wants.
Each command’s flags and arguments become the tool’s JSON Schema parameters.

There’s no second schema to write and then keep in sync (and we all know how well “keep these two things aligned by hand” tends to go). The command tree is the schema. Add a new command to your CLI and it’s a new tool for the agent, automatically, with the description and flags you already gave it. Nobody has to remember to update an MCP manifest, because there’s no separate MCP manifest to forget about.

Configuring an assistant to use it

On the assistant’s side it’s just as undramatic. You tell your AI client (Claude Desktop, Cursor, anything MCP-aware) to launch mytool mcp. From then on the assistant:

Starts your tool in MCP mode when it boots.
Discovers every command as a callable tool.
Calls the right one, with the right parameters, when a user’s request needs it.

Your CLI tool has quietly become something the AI can pick up and use, mid-conversation, on its own initiative.

The safety property worth noticing

Now, “let an AI run things on my machine” is rightly a sentence that makes people nervous. It makes me nervous, and I built the thing. So it’s worth noticing the constraint sitting quietly in this design.

The AI can only call what you defined. The tools it sees are exactly the commands in your tree, and the parameters it can pass are exactly the flags and arguments you declared, validated against the JSON Schema generated from them.

It can’t invent a command. It can’t pass a parameter you never defined. The boundary of what the agent can do is the boundary of what your CLI does, and you drew that boundary already, back when you built the tool. Exposing the CLI over MCP doesn’t widen the surface one inch. It just makes the existing surface reachable. The AI isn’t running things. It’s running your commands, the ones you wrote, tested and shipped, and nothing else.

The gist

A CLI tool, built properly, is already a structured description of a set of capabilities: named operations, descriptions, typed parameters. Which is also exactly what an AI agent needs in order to call a tool. The gap between the two is only a translator, and writing a bespoke one per assistant is a treadmill you don’t need to step onto.

go-tool-base puts the translator in the framework. Every tool gets an mcp command that serves the command tree over the Model Context Protocol… commands become tools, descriptions become descriptions, flags become JSON Schema parameters, with no second schema to maintain. Point any MCP-aware assistant at it and your CLI is an agent-callable tool, bounded to exactly the commands you shipped.

You did the hard part when you built a good CLI. MCP just opens the door you’d already framed.

go-tool-base: I got tired of reinventing the wheel

Wed, 18 Mar 2026 00:00:00 +0000

If you’ve written more than two or three command-line tools in Go, you’ll recognise the shape of the first afternoon. I certainly do! You reach for Cobra for the command tree, Viper for config, and then you start the part nobody ever puts in the README… the plumbing.

Where does config live? A file, an env var, an embedded default? In what order do they override each other? How does the tool tell the user there’s a newer version, and how does it actually update itself? What does logging look like, and is it the same logging the next tool will use? And how do you wire all of that into each command without every command reaching into a pile of globals?

None of it is hard. That’s the problem! It’s not hard, it’s just there, every single time, and every single time I’d find myself reinventing it slightly differently to the last time. Different override precedence here. A subtly different update flow there. Logging that didn’t quite match the tool I’d written three months earlier. Each new tool was a fresh re-litigation of decisions I’d already made and then promptly forgotten.

Now, I’ve banged on about the Boy Scout rule for years (leave the codebase better than you found it), but it has an uncomfortable corollary. If you keep turning up to the same campsite and finding it in the same mess, at some point the honest thing to do is to stop tidying it and go and build a better campsite.

First, just packages

So I started pulling the recurring pieces out into their own packages. Nothing grand. A config package that did the hierarchical merge the way I always ended up doing it anyway. A version package that knew how to compare semver and spot a development build. A setup package that handled first-run bootstrap and self-updating from a release. They lived as separate repos, and if you go digging through my GitHub history you can still find the scruffy ancestors of them scattered about.

Separate packages was the right first move. It forced each piece to stand on its own and earn its keep on a real project before I trusted it on the next one. A package that’s only ever been used in the repo it was born in hasn’t really been tested… it’s just been agreed with.

But separate packages come with a tax. Each one has its own release cadence, its own changelog, its own CI. Worse, they have to agree with each other at the seams, and when they’re versioned independently those seams drift. I’d bump the config package, and the setup package that depended on it would quietly need a matching bump, and the tool that used both would need telling about both. I’d traded “reinvent the wheel” for “keep a dozen wheels in sync”, and I’m really not convinced that’s a better deal.

Then, one library

Once the packages had been used enough (used in anger, on real tools, by people who weren’t me) the shape of them stopped moving. The interfaces settled. The arguments about precedence and defaults were over, because the answers had survived contact with reality.

That’s the point where separate packages stop being a virtue and start being friction. So I forged them into one and called it go-tool-base. One module, one version number, one changelog, and one set of seams that are now internal and can’t drift, because they ship together.

The heart of it is a dependency-injection container, a Props struct, that holds the things every command needs: the logger, the config, the embedded assets, the filesystem handle, the error handler, the tool’s own metadata. Commands are handed Props explicitly rather than reaching for globals, which means a command is just a function of its inputs and is therefore trivially testable. That one decision has quietly paid for itself on every tool I’ve built since.

Around that container sits all the stuff I was so tired of rewriting: hierarchical config, structured logging, version checking, self-update from GitHub or GitLab releases, an interactive TUI documentation browser, AI integration, service lifecycle management. A new tool inherits the lot and gets to spend its first afternoon on the thing that’s actually novel… its own logic.

Finally, a generator

A library still leaves you staring at a blank main.go. You still have to know the conventions, wire the container, lay out the directories, register the commands. All knowable, but all boilerplate. And boilerplate is exactly the enemy I set out to kill in the first place.

So go-tool-base ships a generator. gtb generate project scaffolds a complete, working, idiomatic project: directory layout, the wired Props container, the command tree, CI, the whole lot. gtb generate command adds a new command and registers it for you. The generator also handles upkeep: when the framework’s conventions move, it can regenerate the scaffolding of an existing project without trampling all over the code you’ve written on top. (That last bit turned out to be a properly interesting problem in its own right, and a future post.)

The goal is blunt. Creating a CLI tool should be about the tool, not the scaffolding. The first afternoon should be spent on the part that’s actually worth writing.

One thing I was careful about

There’s a nasty failure mode with “batteries-included” frameworks: the day you outgrow them, they hold you hostage. You either stay inside the framework’s worldview forever, or you face a rewrite. I’ve been burned by that before and I had no intention of inflicting it on anyone else.

So go-tool-base generates idiomatic, standard-library-compliant Go. There’s no magic runtime you can’t see, no clever code you couldn’t have written by hand. If you ever outgrow the framework the generated code stands on its own and you walk away with a perfectly normal Go project. A framework should be a starting point you’re glad you took, not a room you can’t get out of.

Where this leaves me

go-tool-base exists because I was spending the first afternoon of every Go CLI tool rebuilding the same plumbing, and rebuilding it slightly wrong relative to last time. It started life as separate packages so each piece could earn its place on real projects; once they’d stopped moving I forged them into a single library so the seams couldn’t drift; and then I wrapped a generator around it so a new tool starts as a working project rather than a blank file.

It’s a framework for the boring 80% (config, versioning, updates, logging, lifecycle) so you can spend your time on the 20% that’s actually yours.

Over the coming posts I’ll dig into the individual pieces… the generator that won’t clobber your edits, the credential handling, the self-update integrity checks, and a few Go techniques I’m rather pleased with along the way. Stay tuned!