Rewriting go/config around a Store fixed the thing I’d set out to fix, and handed me a much more obvious problem on the way out the door (it was about ten at night, and I’d been feeling rather pleased with myself). The new one read YAML.

Only YAML.

Viper, which I’d just deleted by accident, reads a dozen file extensions. So the objection writes itself before anyone has to be polite about it: a team with a config.json can’t adopt this at all, and telling them to convert their config file to suit my library isn’t a serious answer to anything.

I wasn’t putting five parsers in the core

The obvious fix is five more parsers in the core. Most config libraries in most languages have stood about here, and most of them took that road.

I didn’t fancy it, and the reason was already sat in the estate. Two other modules had solved the same shape of problem… chat has a sibling module per AI provider, forge has one per git host. So the question became how to support the other formats without bloating the core. Adapter projects, one per file type, the way chat and forge already do it. YAML stays the default, and if you want XML you import the XML one and say so.

And then the naming call, which matters a good deal more than it looks. One module per file type, named for it. config-json, config-xml, config-toml. Specifically, and only, to keep the dependency footprint lean.

Why a module and not a package

This bit is peculiar to Go and it catches people out, so it’s worth a plain-English paragraph. Go is good about code. Import a package you never call and the linker drops it; your binary doesn’t carry it. So on the face of it, one big config-formats package with all the parsers in costs a JSON user nothing at runtime.

Except dependencies aren’t worked out per package. They’re worked out per module. One go.mod, one go.sum, listing what the whole module needs. Put five parsers in one module and everybody who imports it inherits all five parsers’ requirements, whether they touch them or not. Their go.sum grows. Their vulnerability scanner has more to say for itself. Their audit has more to read. The binary’s fine and everything around the binary is a bit worse. One module per format is the shape where the graph tells the truth, or at least the only one I could find.

There’s a second benefit I didn’t plan and now rate higher than the first: a format either has a module or it hasn’t. There’s no list of supported formats sat somewhere, slowly drifting out of step with reality.

I found the next bit while I was measuring, and it made me laugh.

What Viper advertises, and what Viper ships

Viper v1.21.0 exports the list of extensions it supports. Twelve of them:

SupportedExts = []string{"json", "toml", "yaml", "yml", "properties", "props",
    "prop", "hcl", "tfvars", "dotenv", "env", "ini"}

Its internal/encoding directory, where the codecs actually live, contains four. dotenv, json, toml, yaml.

HCL is on that list. So are tfvars, INI and Java properties. Point Viper at a .hcl file and it’ll happily accept the extension, get as far as wanting a decoder, and not find one. Now, that’s a maintenance artefact rather than anything underhand, and those formats were almost certainly real once. But it is the failure mode that “a format either has a module or it hasn’t” makes structurally impossible, and I only spotted it because I’d gone looking for a number to stick in a spec.

Right, some actual numbers

Assertions about dependency weight are cheap, so here are some real ones. All the figures come from the same probe: an empty module that imports exactly one thing, then go list.

Two things in there worth saying. The core pulls fewer than half the packages Viper does, while doing considerably more. That’s the result I was after. But it pulls more modules than Viper, because the estate is deliberately built out of small ones. Measure the thing I optimised for and I win comfortably; measure module count and I lose a bit. Both numbers are real, so both go in the table.

The other is that bottom row. Reading config out of Google Cloud Storage costs 455 packages more than the core on its own, and there’s no clever trick that makes that cheaper, because most of it is Google’s client libraries and they are what they are. And that’s rather the point: the only people paying that are the people who asked for it. The spread from config-iofs adding a single package to config-gcp-gcs adding 455 is the whole argument, sat in one table.

One nuance, which I couldn’t put into words for a while. Early on I waved the dependency question away for myself, because keeping libraries current is a normal cost of maintaining anything and Renovate does most of the legwork. Still true. The footprint argument was never about the dependencies I maintain. It’s about the ones I impose on someone else, who may never open that parser as long as they live.

Two things I got wrong, left on the record

The specs carry their own reversals, deliberately unedited, and both are about writers. I’d assumed TOML would need a whole bespoke document module to edit files without mangling them, on measured evidence that no Go library round-trips TOML cleanly. That went down as a decision. Then it collapsed: go-toml/v2/unstable exposes each node’s source byte range, which turns a TOML edit into a targeted byte replacement of about two hundred lines, inline. The module I’d specced never got built.

HCL was the same mistake pointing the other way. I had it down as the hard writer, and it turned out to be the easy one, because HashiCorp already ship hclwrite and it does structure-preserving edits properly. config-hcl shipped reading and writing in one pass. The spec puts it plainly:

inverts the assumption in the first draft of D11, which treated writing HCL as the hard part. It is not; the hard part was always the semantics.

Both mistakes had the same cause. I’d been thinking about the editor as the difficult component, when the parser is the reusable primitive and the editor is thin work on top of it. Get the primitive right and the writer is a couple of hundred lines.

The hard part, as the spec says, was semantics. config-hcl refuses to evaluate anything at all: var.*, ${local.x}, functions, include, all rejected at load and by name rather than resolved. HCL-as-configuration, explicitly not Terraform. Silently resolving half a Terraform file and handing back a plausible-looking answer is far worse than refusing the thing outright.

The format I threw out

HOCON didn’t make it, and the reason is the best argument for the whole design. There’s essentially one usable HOCON parser in Go. Reading it, it hardcodes environment-variable fallback, so a probe with ${LEAKED_SECRET} in a config file resolved it straight out of the process environment. That’s the very behaviour the core has a rule against, for the reasons in the last post about save-your-config writing your tokens to disk.

So the choice was ship a format whose only implementation defeats a core guarantee, or don’t ship it. Viper never advertised HOCON either, so declining costs nothing against any parity claim I might want to make later. It’s out. Meanwhile XML, INI and Java properties ship deliberately read-only, and say so on the tin. A half-working structure-preserving writer that mangles someone’s file is worse than no writer at all, and “we support writing” isn’t a claim I want to make loosely.

Every last one of them

Same decision, made three times over, gave three families. A module per format, a module per filesystem, a module per backend system. Each one gets its own approved spec before anyone writes a line, because the expensive part of a remote adapter is understanding the other system’s semantics rather than writing the Go.

Here’s the lot of them as they stand. Each links to its page on the core docs, because adapters don’t get their own docs site in this estate… they carry a strong README and signpost home.

Twenty-five. That number was twenty-four when I started writing this, because config-etcd shipped while the draft was open! That’s either a decent argument for the architecture or an argument about my inability to finish anything. And the one I was convinced would break the taxonomy didn’t. keryx wanted to keep credentials in the OS keychain, and for a good while I couldn’t have told you which family that belonged to. Looked a bit like a filesystem. Behaved a bit like a backend.

Turns out it’s a backend, and a fairly unremarkable one at that. macOS Keychain, Windows Credential Manager, Secret Service on Linux, sat in the layer stack like anything else, so a refresh token finally has somewhere to live that isn’t a plaintext file sat next to the port number. Three tidy categories, twenty-five modules, and the awkward one filed itself. I’ll take that. The twenty-sixth will be along before I’ve finished the next post, no doubt…