Go on PHP Boy Scout

The library that appointed itself signal handler

Sat, 17 Oct 2026 00:00:00 +0000

A commit had gone into go-tool-base’s root command about a month before I noticed it, and it had been quietly breaking things for anything downstream that used the controls module the whole time. I didn’t find it. Someone raised it with me, which is not the order I’d have chosen, and it’s on me for not spotting it when it went in.

Two right answers, one process

The commit gave the root command a signal-aware execution context: catch SIGINT, cancel the context, let the command wind itself up properly. That’s a good change, and an obviously right one. A CLI wants to flush buffered telemetry before it dies, clean up a half-written self-update, and let a long-running command notice it’s been asked to stop.

I’d have approved it. I did approve it, apparently (without reading it anything like carefully enough), and the trouble is what was already sitting underneath. go/controls is the supervisor that runs long-lived services and stops them in the right order, which I wrote up in lifecycle management for long-running Go services. And for most of its life it caught SIGINT and SIGTERM itself, by default.

That was also correct! A supervisor sat in a bare main really is the outermost thing in the process, and catching the interrupt to drive a graceful stop is its job. Doing it without being asked is a kindness. Two right decisions, eighteen months apart, and a collision that only exists where they overlap.

Nothing errors, which is the problem

Go will happily let two parts of one process both handle Ctrl-C, because signal.Notify is additive: register a second channel and it doesn’t replace the first, it starts posting to both. No error, no warning, no log line saying “you appear to have two of these”. Just two shutdown drivers, racing. So a default about process-global state is less a default about your library and more a claim about your position in someone else’s process, and the post doesn’t arrive telling you when that position has changed.

My own tools had already told me

So I went and looked at what the downstream tools were actually doing, and this is the part that smarted. Two of them had hit this unaided and written essentially the same workaround, with the same reasoning, without ever comparing notes: one had a comment explaining that the framework’s root command already installs the handler that cancels the context so a second one would race it, and the other said it drives its stop from context cancellation specifically to avoid a second handler.

Same conclusion, same boilerplate, arrived at independently by two tools I wrote, working around a problem I’d shipped. And a third had never opted out at all. It had been running both handlers the whole time and I hadn’t noticed, because from the outside this looks like a slightly chatty shutdown.

When two of your users pay the same tax and a third is silently broken, I think you can put the design principles away. The users had already told me the default was wrong, and they’d gone to the trouble of writing it in the comments (my comments, in my tools, which somehow makes it worse).

The part that actually hurt

Duplicated log lines are the visible symptom, and they’re the least of it. controls publishes a contract. When a service’s context is done, context.Cause(ctx) tells you why: if it’s ErrShutdown, your controller stopped you deliberately, and anything else means something upstream died and you should behave differently. That’s how a service decides whether to finish flushing or bail out now.

With two handlers, that becomes a coin flip. WithCancelCause hands you the parent’s cause when the parent cancels first, so if the framework’s handler wins the race, the controller’s own cancel(ErrShutdown) turns up late and does nothing at all.

Same code, same signal, and a different answer depending on which goroutine the scheduler fancied. A published guarantee, broken by scheduling, with no error anywhere in sight. “It worked when I tested it” and “it is correct” have never been the same claim, and this is the gap that lives between them.

Somewhere else in the estate, I’d got it right

What convinced me this was a principle and not a signals quirk is that another module already handled it properly, by default, and had done from day one. go/transport-metrics sets up Prometheus collectors, and its documentation is blunt about it:

The default is a fresh, private registry (never the global prometheus.DefaultRegisterer), so a tool’s metrics are self-contained and tests do not collide.

Different process-global, identical reasoning, opposite instinct. Two libraries in one estate touching shared state, one grabbing it by default and one refusing to, and that comparison did more for the argument than anything I could have written in a spec.

The default went the other way round

NewController no longer installs a signal handler at all, and the doc comment explains itself so you don’t have to work it out:

It does NOT install an OS signal handler. Signal disposition is process-global state and belongs to whichever layer is outermost, typically the CLI framework or main. Pass WithSignals when the controller genuinely is that outermost layer.

The behaviour still exists, you just have to ask for it, and in a bare main where the supervisor really is the outermost thing it’s WithSignals() and you’re away. One small thing I got right by accident: the spec ended up raised against controls and not against the framework, because that’s the module whose default was wrong. A boring filing decision, but it means the reasoning now lives next to the code it governs instead of in the repo of whoever happened to notice.

And keryx’s comment has changed status without changing much text. It used to be a workaround describing what it was avoiding. It now reads as a note citing the decision. Same words, different thing.

The fix I didn’t ship

The obvious worry with handing signals to the CLI is whether the service still gets long enough to shut down cleanly if the framework owns the interrupt. I had a whole cooperative-drain protocol specced out for it. Handshakes, acknowledgements, the lot.

Then I went and read three call sites, and found the graceful path already waits. Wait() blocks RunE, which blocks ExecuteContext, so the root command cannot physically reach its exit path until the controller has fully stopped. The problem I’d spent an evening designing around did not exist, and the only real gap was a single force-exit goroutine on the second Ctrl-C, which is a much smaller creature. An entire protocol, specced and binned, on the strength of going and reading the code instead of reasoning about it.

That’s the second time in this story that the answer was sitting in a file I’d already written, once in my consumers’ comments and once in my own call sites. I should probably read more and design less.

Merged in five days, open in seven hundred

Mon, 03 Aug 2026 00:00:00 +0000

Twice in two days I watched a hint vanish.

The first time was in chat. I found where it had gone, wrote the workaround, made a note in the spec and moved on, because that’s what you do with a small thing at the end of a day. The second time was in forge, the following evening… and a small thing that happens twice is not a small thing any more. So instead of writing the second workaround, I went and read the library.

About four hours later the estate had an errors package of its own, public, with a docs site and a v0.1.0 tag on it. Which is a fair old distance to travel between tea and bedtime, so let me back up a bit.

What a hint is, and where it went

This only lands if you know what a hint is, so bear with me for a paragraph.

Go errors nest. You take a failure, wrap it with a bit of context, wrap that with a bit more, and what comes out the far end is a little chain you can walk back down. cockroachdb/errors, which the whole estate has been built on since it was one framework, lets you hang extra things off that chain. A hint is the useful one: not what went wrong, but what the person at the keyboard should actually do about it. Something fails four layers down, the code that catches it walks the chain, collects the hints, and the user gets told to import their keys rather than just “permission denied”. I wrote about that idea back when the estate first leaned into it, and it’s the one thing I really didn’t want to give up.

Go 1.20 added errors.Join, which bundles several errors together into one. The standard library gave those bundles a shape you can recognise on sight: an Unwrap that hands back a slice of errors instead of a single one. Plural. Go and look in all of them.

cockroachdb/errors is older than that by some years, and its own Join puts a stack-capturing wrapper around the bundle. So from the outside, a bundle looks exactly like any ordinary one-link error. Anything walking the chain arrives, sees a single link, follows it, and stops. Everything below the join is invisible: hints, details, context tags, telemetry keys, the lot.

Which, it turns out, somebody had already gone to the trouble of reporting. Issue #162, raised on the 14th of May, and they put it more precisely than I would have. The standard library lets you do this:

err.(interface{ Unwrap() []error }).Unwrap()

and with cockroachdb/errors you need this:

err.(interface{ Unwrap() error }).Unwrap().(interface{ Unwrap() []error }).Unwrap()

An extra hop, through a wrapper nobody asked for. And it does it even when you join a single error, which is the part that actually bit me… code that behaves itself all day breaks the moment it happens to take the aggregate path with one thing in the bag.

It isn’t a bug, and that’s worse

So I wasn’t the first to find it, which was a small comfort right up until I looked at the date again. Getting on for three months, and the issue is still open. Nobody has replied to it at all.

One stale issue tells you nothing, mind. Every project has those, mine included. So I went and read the rest of the tracker.


#162, the defect above	open since May, no reply
#147, a one-character docs typo	open since 3 August 2024
#129, `slog.LogValuer` support	open since 2023
#159, a compatibility fix that broke CockroachDB’s own build	merged in five days

That last row is the whole story. Five days! Because it was breaking something CockroachDB needed.

Which is not neglect, and I want to be careful here, because it would be very easy to write the angry version of this post and the angry version would be wrong. That library is maintained, openly and perfectly rationally, for one consumer’s needs. Things CockroachDB needs move in days. Everything else waits years. That’s a perfectly sensible way to run a library and a poor thing to build fifty modules on top of.

Half the reasons I picked it

I chose cockroachdb/errors for exactly two reasons, and I know that because I wrote them down at the time: the feature set, which was enormous, and the fact that it was well maintained and I could rely on it.

The features held. Every one of them is still there, still working, still better than what I’ve replaced them with, if all you’re counting is features. It is the second reason that quietly stopped being true, and half a premise is not a premise.

I have been here before, and I picked the same way

Errors are the fourth thing go-tool-base has stood on, and if you find yourself choosing an error library for the fourth time you probably ought to be a bit suspicious of yourself.

It started where everything starts, on the standard library. Then go-errors/errors, six years or so back, in the days of gtb’s various predecessors. It was great when I picked it up… then it got old, rather quickly… and then it stopped really being looked after at all. By the time gtb was taking its current shape I could feel it holding me back, so I went hunting for something newer, more mature, better maintained.

That hunt was about eighteen months ago and it came down to two. pkg/errors was the obvious candidate, and I passed on it. Not on features. On its long backlog of open issues. cockroachdb/errors looked like the better-kept project of the pair, so that is where the estate went, and where it has been ever since.

Go back and read that again, because I have, a few times. Eighteen months ago I rejected a library because too many of its issues were sitting unanswered, and picked the one that looked healthier. And I have just left that one for the same reason. Same test. Same instinct. Different answer.

Which isn’t a contradiction, and I’m not embarrassed by it. That’s all a dependency choice ever is: a bet you place on the evidence in front of you, on the day you place it. It was the right call. Things move.

What has changed is what it costs to be wrong. That migration was done by hand, find-and-replace, no agent to help me. Dull and slow, but gtb was still a single monolithic framework back then, so it was one codebase and I could kinda hold the whole shape of it in my head. This time it’s fifty-plus modules, every last one built on the assumption that cockroachdb/errors is simply what we use. That’s not a job you take on lightly, agent or no agent.

The bar, and whether this clears it

Now… the bit I’m touchiest about.

I cannot stand “Not Invented Here”. I call it out when I see it, probably harder than is entirely fair. And I’m well aware that a bloke with fifty modules to his name, announcing he’s gone and written his own errors package, is on fairly thin ice here.

So there’s a bar, and everything I’ve built has had to get over it before I’d write a line:

It addresses a need nothing out there is meeting.
It brings something genuinely new.
I can realistically expand and maintain it.

chat clears it: a much lighter-weight alternative to langchain, which is a wonderful thing and absolutely enormous. config clears it, on provenance, layering and true writeback. forge clears it, because the forge SDKs are wildly inconsistent where they exist at all. What you don’t see is the pile that didn’t clear it, and there is a pile. Only recently I put proper time into investigating a proposal, wrote the thing up, and then binned it, because it didn’t satisfy a single one of the three.

And in the interests of not marking my own homework too generously: scoutdm exists largely because I wanted it to exist. That’s hubris and I know it. Only time will tell whether I was right.

So. Does an errors package get over it?

The first one, yes, and I went and checked properly rather than assuming, because assuming is exactly how you end up rebuilding something that already exists. Every alternative is a worse maintenance bet than the thing I’m leaving, and not one of them has a hint concept at all. The closest, tozd/go/errors, is genuinely good and I want to be fair to it: it takes pkg/errors, keeps the API, and fixes a long list of things the archived original was never going to get to, because the fixes needed breaking changes. It even keeps a pkg/errors import on purpose, so you can mix the two while you migrate off. There are just no hints in it, and hints are what I came for.

The third one, comfortably. The core is 476 lines, and the whole of its go.mod is this:

module gitlab.com/phpboyscout/go/errors

go 1.26.5

Not “few dependencies”. None. And that’s asserted by a test rather than left to my good intentions, because every module in the estate imports errors, so whatever it drags in, they all drag in. That’s exactly how the library I’m leaving came to contribute forty-six of forge’s three hundred and seventy-six transitive packages, a deprecated gogo/protobuf among them, in return for the ten or so functions I actually call.

The second one is where it wobbles, and I’d rather say so myself than have you spot it. There’s nothing novel in here. It’s a small, dull errors package that does what the estate needs and then stops. Two out of three, and the one it’s short on is the one I’d hold somebody else to.

Where I left the holes

The one thing I did dig my heels in about, before a line was written, was where the gaps go. I’ve paid for this migration twice now and I’d quite like the third to be the last.

Wire serialization was a headline reason I picked cockroachdb in the first place. Nothing I run today needs it, but plenty of what I want to build will, and losing the ability to add it later would make this whole move a downgrade. So it comes later, as a sibling module, and the core is already shaped to take it: every layer carries a stable kind, and nothing anywhere type-switches on a concrete type. Which means a decoder that meets a kind it has never heard of can still carry that layer along without snapping the chain.

OpenTelemetry got the better question, and I asked it against my own proposal: should an error fire an OTEL call at all? An error that reports itself is doing two jobs and holding opinions about your observability stack. Deciding a thing is worth recording belongs where a span is in scope, not where the error got made. So the package ships no RecordError, only the contract you’d build one from. go/observability already owns that relationship and it can keep it.

Not YAGNI, and not building a cathedral for an imaginary congregation either. Somewhere in between: work out which of the things you might want later would force you to tear it up and start again, and leave holes the right shape for those. Those only.

The cutover is running as I write this. Leaves first, working upward, which isn’t the order you’d guess, and is a story for another day.

It lives at gitlab.com/phpboyscout/go/errors, with docs at errors.go.phpboyscout.uk. MIT, no dependencies, and it kept the cockroachdb names on purpose, so if you happen to be in the same boat the move is mostly one import path and a sed. I make no grander claim for it than that it’s what my estate runs on now, which is either a recommendation or a warning depending on how the next year goes.

cockroachdb/errors is still there, and it’s still good. Still maintained exactly as carefully as it was last week, by somebody doing a perfectly reasonable job for the people he’s doing it for. That one-character docs typo is two years old today, to the day, and nobody is going to mark the occasion. Somebody is still waiting on it, and none of that is my problem any more.

I just stopped being one of the people it was for.

The version pin you can't lower

Sun, 02 Aug 2026 00:00:00 +0000

A security fix landed in the Go standard library, the kind you don’t get to think very hard about. You bump go.mod to the patched toolchain, 1.26.4, and you move on. Or you would, if your release didn’t immediately fall over the moment you tagged it.

Here’s what greeted me when I cut the next krites release. The before-hooks, the go mod tidy and friends that run ahead of the actual build, died on the spot:

go.mod requires go >= 1.26.4 (running go 1.26.3; GOTOOLCHAIN=local)

The goreleaser image I was releasing from, a perfectly current v2.16.0, bundled Go 1.26.3. One patch version behind. And GOTOOLCHAIN=local, which is the sensible default for a release because it means “use exactly the Go that’s installed, don’t go fetching anything mid-build”, was doing precisely its job: it saw go.mod asking for a toolchain it didn’t have, and refused to pretend otherwise.

Now, the fix that’s quickest to reach for is the one you have to talk yourself out of. The error is a version mismatch, and the fastest way to make a version mismatch go away is to change one of the versions. The image has 1.26.3? Fine, drop go.mod back to 1.26.3 and the red goes green. Except think about what that actually does. The whole reason go.mod says 1.26.4 is that 1.26.4 carries a security fix I need. Lowering the pin to placate the build image doesn’t resolve the tension, it surrenders to it. It un-applies the security update to make a pipeline pass, which is about the worst trade in software, and it’s the one sitting right there with a green tick on it.

So that pin is one you can’t lower. Which leaves making the build find the right toolchain instead of making the requirement smaller.

The quick version is one flag: flip GOTOOLCHAIN from local to auto. Where local means “only ever use what’s installed”, auto means “if go.mod asks for a newer toolchain than you’ve got, go and fetch that one and use it for this build”. So the release runs on the image’s 1.26.3, hits the go.mod directive, fetches 1.26.4 and carries on. The mismatch is resolved by upgrading at build time rather than by downgrading the requirement, which is the right direction for it to resolve in.

The better version, which is where krites ended up, is to not have the gap at all. The release now runs on the dev-tools image I bake my CI tools into, and that image tracks the Go version my code actually targets, 1.26.4. When the toolchain in the image and the toolchain in go.mod are the same thing by construction, there’s nothing to fetch and no flag to set, because there’s no disagreement to paper over.

The toolchain your code requires and the toolchain your build image ships are two different facts, and they drift apart every time a point release with a security fix lands in one a good while before it reaches the other. When they disagree, the instinct to make them agree by lowering the requirement is exactly backwards: the requirement is the thing that’s right, and the environment is the thing that’s behind. The pin only ever moves one way. The build image is what had to catch up to it, not the code learning to want less.

A mutex on a flag nobody writes twice

Thu, 16 Apr 2026 00:00:00 +0000

“Why is there a mutex around a boolean that only ever gets set once?”

It’s a fair question, and I’d half-asked it of myself before someone asked it of me. The answer turns out to be written, in as many words, in a code comment I’ve grown rather fond of.

The registry and its one-way latch

go-tool-base keeps a feature registry: the initialisers, sub-commands, flags and checks that each feature adds to the CLI. Features register themselves into it at startup, from init(), before main runs. Once everything’s wired, the framework calls SealRegistry() and the registry latches shut. Any Register call after that point panics, on purpose, because a sub-command or flag that turns up after the CLI has parsed its arguments is a bug I want to hear about at once, not discover three releases later.

So there’s a registrySealed bool. It starts false, SealRegistry flips it to true exactly once in normal operation, nothing flips it back outside of tests, and it’s read on every registration attempt. Written once, read many. The textbook shape of “you don’t need a lock for this.”

Except the comment disagrees, on purpose

Here is the actual declaration, in pkg/setup/registry.go:

// registryMu protects globalRegistry and registrySealed. Acquired for write
// by all Register* and Reset/Seal helpers; acquired for read by all Get*
// accessors. The mutex is required for memory visibility of registrySealed
// across goroutines, not only mutual exclusion on the maps.
var (
	registryMu     sync.RWMutex
	registrySealed bool
)

That last sentence is the entire post. The mutex has an obvious day job: the registry is a clutch of maps that get appended to during registration, and concurrent appends need genuine mutual exclusion. registrySealed could have just hitched a ride on that lock and nobody would have thought twice. But the comment goes out of its way to say the lock is also required for the flag, for visibility, not only exclusion.

Why a write-once bool still needs the lock

The Go memory model makes no promise that a goroutine reading registrySealed will ever see the write SealRegistry made, unless there is a happens-before relationship between them. No synchronisation, no guarantee. A reader can sit there seeing false long after the seal happened on another goroutine, because the compiler may cache the read and the CPU may serve it from a core-local view. And a concurrent read and write of the same variable, with nothing ordering them, isn’t “probably fine”; it’s a data race, which Go defines as undefined behaviour.

“But registration is single-threaded, it’s all init().” It was, right up until we wanted the tests to run in parallel. This lock exists because of a deliberate campaign to restore t.Parallel() across the codebase after a stack of races forced us to drop it (the same campaign that retired the package-level mocking hooks). Tests build, register, seal and reset this registry from parallel goroutines. The instant that’s true, the seal check has to stay correct while racing, because the very thing it guards against is concurrency. So reads take registryMu.RLock, the write takes registryMu.Lock, and now there’s a happens-before edge: anyone who acquires the lock after SealRegistry released it is guaranteed to see true.

What the lock is actually for

It isn’t there to stop two goroutines both sealing the registry. There’s only ever the one seal. It’s there so that every reader can trust what it reads. A value written exactly once is precisely the case where you’re most tempted to skip the synchronisation, and precisely the case where skipping it can leave a reader legally staring at the stale value for good. The comment spells it out so that the next person to glance at registrySealed, think “that clearly doesn’t need a lock,” and reach for the delete key, reads the sentence first.

(There’s a sibling sealed flag in the middleware registry that follows the identical pattern, for the identical reason.)