Featured image of post My reason for hand-rolling it was wrong

My reason for hand-rolling it was wrong

I wrote my own OpenPGP packet code, then deleted the lot to prove I needed it. The reason I'd given turned out to be false, and a better one was underneath.

One evening I deleted four hundred lines of my own cryptographic packet code, replaced the lot with someone else’s library, and ran the tests. They passed, which was not the result I was after.

Writing your own is the worst default there is

The argument starts against me, so I’ll start there too. There is a maintained OpenPGP implementation for Go. ProtonMail/go-crypto has something like 479 known importers, including go-git, Terraform, OpenTofu, Flux and most of the release-signing tooling in the ecosystem. If you need OpenPGP in Go, that’s the answer, and writing your own packet, signature and MPI code instead is close to the worst default a project can pick.

I’d written my own anyway, in go/encryption, and I had a reason. Our private key lives in a KMS, so it cannot be read, exported or handed to a library. Assembling an OpenPGP certificate needs signatures made with that key, and my claim was that go-crypto simply couldn’t do that, because it wants a private key and I haven’t got one to give it. Reasonable, confident, written down in my own docs. And wrong!

Test it by deleting it

I didn’t argue about it, I deleted the package. Actually removed it (not “decided not to use it”, gone), wired go-crypto in its place, and ran the acceptance tests to see what broke.

As far as I could tell, nothing broke.

packet.PrivateKey accepts a crypto.Signer in place of an actual key, so a signature can be made by something that never exposes any key material, which is more or less the definition of a KMS. A spike carried it end to end: a full certificate, both signatures made through a signer, accepted by an independent parser. So the four hundred lines went, because a claim that doesn’t hold isn’t much of a justification, it’s a story I’d been telling myself.

There was a bonus humiliation buried in the diff, too. Somewhere in that deleted code sat a real bug of mine: an MPI header written before the leading zero octets were stripped, corrupting roughly one certificate in every hundred and twenty-eight. A one-in-128 bug, in cryptographic code I’d written because I didn’t trust the alternative!

Then two things refused to die

The interesting part is what happened next, because the package came back. Not all of it, and not for the reason I’d originally given.

The first thing that held is that ECDH decryption has nowhere to plug in. go-crypto’s encrypted_key.go hard-asserts *ecdh.PrivateKey when it decrypts a PKESK packet, so there is no interface to implement, no crypto.Decrypter equivalent for ECDH, and no way to hand it a shared secret computed somewhere else. A key that lives in a KMS cannot be used for that at all, and I do mean at all rather than awkwardly or slowly.

That’s a real asymmetry worth sitting with: RSA has an interface for this and ECDH hasn’t. Signing was fine because signing had crypto.Signer; decryption wasn’t, because the equivalent hadn’t been built (yet, I suspect, and there is an open request for it).

The second thing that held is the dependency footprint. go/encryption’s build graph contains nothing but the standard library, and a test asserts it so nobody has to remember. Pulling go-crypto in costs golang.org/x/crypto, github.com/cloudflare/circl and everything underneath them, and when that dependency went in during the experiment it brought eighteen CVEs along that had to be resolved by raising pins. A consumer whose entire need is “decrypt a message addressed to my KMS key” would inherit the lot.

This is not go-crypto’s fault

There’s an open request from April asking for exactly the ECDH interface I needed. A maintainer’s reply is worth quoting:

For RSA decryption with a hardware token, it’s possible to set a crypto.Decrypter as the PrivateKey […] I would prefer to do the same for ECDH.

Agreed in principle, reviewed inside a day, then silence for four months including an unanswered follow-up. A related PKCS#11 request was declared out of scope outright.

That reads like neglect until you remember what Proton actually is. Proton Mail holds its users’ keys locally, under a passphrase. Externally held keys are not their problem to solve, and a maintainer declining hardware-token support for a product with no hardware tokens is making a perfectly sound call about their own roadmap. That’s the thing I’d been mis-describing all along: go-crypto isn’t an abandoned library so much as a vendor library that the rest of us defaulted to.

How we all ended up here

The history is worth the detour, because it explains the incentives. golang.org/x/crypto/openpgp was frozen and deprecated in March 2021. The proposal is blunt about why: it was “effectively unmaintained” and “the crypto package with the most open CLs”. The Go team had already tried to recruit a community maintainer in 2019 and no one appeared. The freeze notice suggests using a maintained community fork, and then adds, carefully: we don’t endorse any specific one.

Proton had already forked in 2019, because they needed OpenPGP for Proton Mail and upstream had stalled, and the other serious fork, keybase’s, was archived in 2021. So the ecosystem converged on go-crypto by attrition, the one still standing after the others had gone.

Half the Go release-signing tooling on the planet now depends on a fork maintained, entirely legitimately, for one company’s mail product. I can’t point at a wrong decision anywhere in that chain. It’s just what happens when a language freezes a package and hopes.

Why assembly came back too

I’d have got this bit wrong if I’d stopped at “keep decryption, delegate assembly”. Assembly and decryption are one problem, not two. The key-derivation function binds the recipient’s fingerprint into its parameter block, so a certificate assembled by one implementation and a message decrypted by another have to agree on that value exactly.

Split them across modules and neither half can be tested without the other, and the acceptance test that actually matters (encrypt to a certificate we assembled, then open it) ends up living in a third place dragging both in.

Keeping them together costs about three hundred lines and buys a test that runs on every commit with no credentials at all. That’s a good trade, and it’s a completely different argument from the one I started with.

What it looks like now

go-crypto is still in that go.mod, as a test dependency. Each message the package can open, it opens alongside go-crypto and checks they recover the same session key, and a second suite runs the same thing against gpg itself, because gpg is what a security researcher is going to have on their machine. Hand-rolled implementation, validated against the library I didn’t use and the binary everyone else has.

So I’ve ended up somewhere I didn’t expect. The code survived. My reason for writing it did not, and I only found that out by deleting the lot and watching the tests go green… four hundred lines gone, three hundred back, and a justification that’s now narrow enough to be true.

Built with Hugo · Theme Stack designed by Jimmy