There’s a file on your machine that a person wrote by hand. A config file, probably. It has comments in it, because whoever wrote it knew the next person would need them: why the port is that port, which flag needs a firewall change, a note to keep a list alphabetical. The blank lines are where they are on purpose. So is the alignment of the inline comments, because a person lined them up.

Now change one setting in that file with a program, and look at the diff.

I’ve spent most of a weekend on that diff (it’s the Monday after as I write this, and I’m a tad cross-eyed), so bear with me while I explain why.

Two jobs, one word

“A YAML library” covers two different tools that happen to share a file format, and most of the fickleness people complain about (and YAML is fickle, in every language I’ve used it from) comes from using the one for the other’s job.

A value library turns YAML into Go values and back. It’s what yaml.Unmarshal into a struct is for, it’s fast, and it’s lossy by design: comments, key order, quoting, blank lines and layout have no representation in a map[string]any, so they can’t survive one. That’s fine for data a machine wrote and a machine will read.

An editing library changes one thing in a document and leaves the rest exactly as the author wrote it. That’s what a config set command needs, and a settings screen over a file people also edit by hand, and a bot that bumps a version across two hundred repositories, and anything at all where the change is going to be reviewed. Every other ecosystem I can think of keeps the two apart: Rust has toml and toml_edit, HashiCorp has hclsyntax and hclwrite, Python has tomli and tomlkit. Go had the value half many times over and the editing half not at all.

I needed a rock solid YAML editor, not a values library. So I built one. It’s called yamldoc, the docs site is the place to start, and the rest of this post is why it exists.

Built on someone else’s assumptions

The first version wasn’t a library of its own. It was the document layer of go/config’s Store, the bit that writes a value back to your file, spun out into its own module the same day it was written, and it was functional… but not perfect. It sat on goccy/go-yaml, which is a fantastic parser and, for a while, the best syntax tree of the bunch. It also had 149 open issues and 70 open pull requests at the time of writing, with issues going back to 2020 and pull requests waiting since 2022, so the odds of getting a contribution in were starting to look slim. There are more active forks, of course, and I’d have happily lived on one of them if the problem had been a bug.

The problem was an assumption. Comments in a file were effectively ignored… not discarded, goccy is better than that, just ignored. They get attached to the nearest token and carried along in the hope they’ll land somewhere sensible on the way out. Every editing feature I wanted was built on someone else’s idea of what an editor needed, and I needed more than that: higher fidelity, more opinionated in the places I cared about and less in the places I didn’t, and it absolutely had to honour the YAML specifications rather than one library’s reading of them.

Comments should be first-class citizens of a file, not an afterthought. YAML isn’t just a machine-readable format, it’s meant to be human-readable too… why else would a serialisation spec include comments at all? For pure machine readability, well, that’s what JSON is for!

So over a weekend (a long one, more on that in a minute) the wrapper became something I wasn’t prepared to keep, and yamldoc got an engine of its own. No parser underneath it. It’s spec 0008 if you want the hundred and seventy-odd decisions that took, and I suspect that’s a post in itself.

What it does to a file

Let’s take a file. A small one, with the things a person puts in a file: a comment explaining a decision, two inline notes lined up so they read as a column, and a blank line between the sections because they are different sections.

# Which port the public listener binds to.
# Changing it needs a firewall change too.
server:
  port: 8080        # not 80, we sit behind a proxy
  host: localhost   # loopback only in dev

# Feature flags. Keep alphabetical.
features:
  beta_ui: false

Now the job. A user runs myapp config set server.port 9090. One key, one new value, and the file should come back with one line different. I ran that edit through three libraries, four ways, each through its own API (the harness and the exact versions are linked after the outputs), and what follows is their output, byte for byte.

The official library, the map route

go.yaml.in/yaml/v3 is the YAML organisation’s own Go library (yaml/go-yaml), the continuation of the gopkg.in/yaml.v3 everybody learned on, which is archived now; the two produce identical bytes here, and so does the v4 release candidate. The usual route is yaml.Unmarshal into a map, change the value, yaml.Marshal back out. At its defaults you get this:

features:
    beta_ui: false
server:
    host: localhost
    port: 9090

The port changed, so in one sense it worked. Everything else is gone. Both comment blocks, the two inline notes, the blank line. The indentation has doubled, because four spaces is the marshaller’s default. And look at the order: features now comes before server, and host before port, because a map has no order and the marshaller sorts keys alphabetically on the way out. The author wrote the port first because it’s the one you change; the library has an opinion about that and it wins. This is what “lossy by design” looks like on a real file, and I’d stop short of calling it a bug, because it’s a value library doing what it’s for.

The official library, the node route

The same library has a second door, yaml.Node, which keeps comments. Walk the tree to server.port, change the value, encode. With the indent set to two, because at its default this route also re-indents every line to four, you get this:

# Which port the public listener binds to.
# Changing it needs a firewall change too.
server:
  port: 9090 # not 80, we sit behind a proxy
  host: localhost # loopback only in dev
# Feature flags. Keep alphabetical.
features:
  beta_ui: false

Better. The order held and the comments survived. But the column of inline notes has collapsed to a single space, because the emitter has one rule for where a comment goes and applies it to every line, touched or not. And the blank line between the sections has vanished, because a blank line isn’t a node and the tree never had it. A reviewer looking at this diff sees four lines changed for a one-line edit, three of them in places no one asked for.

goccy, through its path API

goccy/go-yaml is the library the first yamldoc sat on. Through its path API it does this:

# Which port the public listener binds to.
# Changing it needs a firewall change too.
server:
  port: 9090
  host: localhost # loopback only in dev

# Feature flags. Keep alphabetical.
features:
  beta_ui: false

The blank line survived, which is why goccy was the best of the bunch for a long while. But the inline note on the line we edited has gone altogether, # not 80, we sit behind a proxy, which is the very comment the next person needs when they come to look at that port. And the note on the line below lost its alignment. This is the “ignored, not discarded” problem from earlier: the comment is attached to a token, the token got replaced, and the comment went with it.

yamldoc

And yamldoc gives you this:

# Which port the public listener binds to.
# Changing it needs a firewall change too.
server:
  port: 9090        # not 80, we sit behind a proxy
  host: localhost   # loopback only in dev

# Feature flags. Keep alphabetical.
features:
  beta_ui: false

One line changed, and only the four characters of the value on it. The comment on that line is where it was, still aligned with the one below. The blank line is there. The order is the author’s. Run a diff and the diff is the bump.

Why it comes back like that

The reason isn’t a longer list of special cases. The others parse into a tree and emit the tree, so whatever the tree doesn’t model is gone and the emitter’s opinions apply to every line. yamldoc edits the source: the bytes inside the edit’s footprint get replaced (here, 8080), and every other byte is handed back untouched. There’s no list of preserved constructs because there’s nothing that isn’t preserved.

Then it checks its own work. Every transaction re-parses the bytes it just produced, composes them fresh, and compares the result against a statement of what the edit meant that was written down before the writer ran: the values, the correspondence between old and new, the untouched bytes, and which comment belongs to whom. Only if all of that agrees does the new revision get installed. A bug in the writer gives you an error and an unchanged file, not a damaged one. I’ve had the damaged one (more than once, if I’m honest) and I’d take the error every time.

That’s one file, hand-picked to make the point. The measured version runs every library that can round-trip a document over eight real project config files, first with no edit and then setting each of seventy reachable scalars through the library’s own API:

• Files back byte-identical with no edit: yamldoc 8 of 8, the next best 5.
• Single-key edits that changed one line and one line only: yamldoc 70 of 70, the next best 24.
• The official YAML test suite: all 306 valid cases parsed and all 94 invalid ones rejected. 304 match the suite’s pinned events and the other two match libyaml instead, and I’d rather tell you that than round it up. I did ask whether we could close the gap. The answer was no, and it’s good enough now we can explain it.

The harness, the versions and the raw output are in the comparison report, measured on the 13th of September, and the construct-by-construct table (CRLF, emoji, a comment before ---, a missing final newline, and the rest) is on the Should you use yamldoc? page, which is the plain version of that question, including the bit where the answer is no.

The four outputs above came from this harness on Go 1.27.1: go.yaml.in/yaml/v3 v3.0.5 (the archived gopkg.in/yaml.v3 v3.0.1 and the go.yaml.in/yaml/v4 v4.0.0-rc.6 candidate return the same bytes on this file), goccy/go-yaml v1.19.2 and yamldoc v0.6.1. Run it yourself, that’s rather the point.

What it costs

In the same breath, because it isn’t free. An edit costs four to seven times what the value libraries charge, and a decode about three times, because yamldoc keeps everything (every token, the whitespace, the comments, the spelling of every number) and then pays again to check the result. A parse is level with goccy. And a parse followed by writing the file back with no change is free, level with the fastest of them, because it never re-emits: it hands you the bytes it started with. Why an edit costs more is the accounting, stage by stage, and the instruction I gave for that page was to justify the expense against the safety, not to hide it.

What it cost me

The weekend, mostly, and ninety per cent of an OpenAI Pro subscription overnight. One run got carried away, seven hours of tokens on scaffolding and tests and precious little engine, because I hadn’t given it enough direction to know when to prefer implementation over testing. That’s on me, and it’s a war story for another day. I don’t think it was wasted, though… it was a real push of a new model and the results speak for themselves: a rock solid spec and testing out the wazoo. The bill for what it costs me long-term hasn’t come due yet… I’m happy with it so far.

Who it’s for

keryx uses it directly, to save its workspace file without wrecking what a person typed into it. Perhaps more to the point, any project built on go-tool-base relies on yamldoc as a default part of its configuration management, so as I carry on building cool shit, more and more of the estate ends up standing on it whether it knows or not. That’s the reason it has bounds on everything (bytes, nodes, depth, alias expansion, work per transaction) and refuses a hostile file with an error rather than a large allocation.

It is not a struct decoder and it won’t become one. Use it with a value library, not instead of one. If you’re reading YAML into a struct, go.yaml.in/yaml/v4 or goccy is the right tool and yamldoc will only get in your way. If you’re changing a file that a person maintains, this is the one. The docs are at yamldoc.go.phpboyscout.uk, getting started says to allow fifteen minutes, and the install is the usual:

go get gitlab.com/phpboyscout/go/yamldoc

f, err := yamldoc.Parse(src, yamldoc.Options{})
if err != nil {
    return err
}

d, _ := f.Snapshot().Document(0)
if err := d.Set("server.port", 9090); err != nil {
    return err
}

out, err := f.Snapshot().Bytes()

Parse bytes, change one key, write bytes back. The file comes back as its author wrote it, comment and all, and I’d suggest that’s what the spec had in mind when it gave you comments in the first place.