Aws-Kms on PHP Boy Scout

Sign your own binaries with go-tool-base, part 7: rotation and break-glass

Tue, 23 Jun 2026 00:00:00 +0000

Most signing guides stop the moment the first release goes out the door, which is a shame, because the question that keeps you up at night comes later: what do you do when the key has to change? Keys get rotated on a schedule, keys get compromised, and one day you’ll want to move off RSA onto something newer. This last part of the signing series covers the bit everyone skips, swapping a signing key out from under a fleet of installed tools without locking a single one of them out, and the break-glass key for the day the primary is gone.

By Part 6 you’ve a pipeline signing every tagged release through a KMS key, a public key published over WKD, and that same key baked into the binary as its trust anchor. It all works. The catch nobody mentions is that you’ve now got a key embedded in software sitting on other people’s machines, and that key is the one thing in the whole chain you can’t quietly change. So let’s plan for changing it properly.

Why there’s no auto-rotate button

If you’ve used KMS for encryption, you’ll know it can rotate keys for you on a yearly tick. Asymmetric SIGN_VERIFY keys don’t get that, and the terraform-aws-signing-kms module sets enable_key_rotation = false on purpose. That’s not an oversight to work around; it falls straight out of how the key is built. The private half of a signing key never leaves the HSM and has no export path at all, so there’s no mechanism by which AWS could hand your verifiers a rotated public half and keep the old one verifiable. KMS auto-rotation works for symmetric keys precisely because you never see the key material; the instant you need a public key your clients pin against, rotation stops being something a cloud provider can do behind your back.

So rotation here is a runbook you maintain, not a checkbox you tick. That sounds like the worse deal until you weigh it: you’ve got a key with no exfiltration path, in exchange for rotating it by hand on the rare occasions you must. For a release-signing key that’s exactly the right trade. You don’t want this key rotating itself; you want to be standing right there when it happens.

Rotate by minting a new key, never by changing the old one

Here’s the rule that makes everything else fall into place. You do not edit the existing key. The module’s key_spec and name are immutable by design, and that immutability is a feature: the v1 key is a fixed point that stays verifiable while you stand up its replacement next to it.

So you mint a second key. A second module instance, named acme-release-signing-v2, with its own alias alias/acme-release-signing-v2:

module "signing_kms_v2" {
 source = "gitlab.com/phpboyscout/signing-kms/aws"
 version = "0.1.2"

 name = "acme-release-signing-v2"
 oidc_provider_arn = data.aws_iam_openid_connect_provider.gitlab.arn

 ci_subject_filters = [
 "project_path:acme/acme-cli:ref_type:tag:ref:v*",
 ]

 key_administrator_arns = [/* ... */]
 automation_role_arn = data.aws_iam_role.automation.arn
}

Apply that and the v1 key carries on signing, undisturbed. Now mint its public half exactly the way you minted v1 back in Part 4, only the alias changes:

gtb keys mint \
 --backend aws-kms \
 --kms-region eu-west-2 \
 --key-id alias/acme-release-signing-v2 \
 --name "Acme Releases" \
 --email release@acme.dev \
 --created "2026-06-08T00:00:00Z" \
 --output signing-key-v2.asc

You’ve now got two real signing keys, both live, neither one a threat to the other. The whole rotation is the careful business of moving traffic from one to the other while every installed client keeps verifying.

The dual-publish, dual-sign window

This is the part that does the work, and it leans on one fact about how the verifier treats your keys. The trust anchor isn’t a single key, it’s a set: v1, v2, and the rotation authority all sit in internal/trustkeys/keys/ and all get embedded together. A release verifies if any key in that set validates its signature. That’s what makes a handover window possible, because during it a client might hold v1, or v2, or both, and verify happily whichever it has.

So you run an overlap. Three moves, in order.

First, publish both public keys over WKD. Same email bucket, same command you already know from Part 4, just with v2 added to the file list:

gtb keys wkd \
 --domain acme.dev \
 --email release@acme.dev \
 --output ./wkd-staging \
 signing-key-v1.asc signing-key-v2.asc rotation-authority.asc

Both signing keys share release@acme.dev in their UID, so they land in one hu/ bucket together, and the rotation authority rides along as it always has. Deploy that staging tree the way you deployed it before. Now the WKD endpoint serves the new trust anchor, so a client that fetches keys on its next update picks v2 up automatically.

Second, embed v2 alongside v1 and ship a release. Drop signing-key-v2.asc into internal/trustkeys/keys/ next to v1, cut a release, and that build now ships knowing about both keys. Installed clients pick the new trust anchor up as they update through the window. This is the slow bit, and it should be: you want the new key spread far and wide before you stop signing with the old one.

Third, sign with v2, wait, then retire v1. Point the pipeline’s GTB_SIGNING_KEY_ID at alias/acme-release-signing-v2 so new releases are signed by the new key. A client that’s already updated verifies against the embedded v2; one that’s lagging still has v1 in its set and, because you’re still publishing v1 over WKD, can still fetch and trust it. Nobody’s locked out.

Leave that overlap running long enough that you’re confident the slow movers have updated, weeks, not hours, depending on how often your users actually run the thing. Only then do you retire v1: drop its module instance, pull signing-key-v1.asc from the embed directory and the WKD file set, and ship a release that knows only about v2. The rotation is done, and at no point did an installed tool see a signature it couldn’t check.

The break-glass key, for when there’s no handover

Everything above assumes the old key can hand over to the new one: it’s still there, still able to sign, and you’re rotating on your own terms. The nightmare is the other case. The KMS key is gone, access revoked, or you’ve reason to think it’s compromised and you daren’t sign anything with it again. There’s no handover, because the thing that would do the handing is exactly what you’ve lost.

That’s what the rotation-authority key is for, and it’s why Part 4 had you mint it the moment everything else was calm. It’s an offline ed25519 key, generated once on a trusted machine, and the go-tool-base how-to, generate the rotation-authority key, walks the storage in detail. The short version: the private half never lives on a networked box. You print a paper backup with paperkey, write it to an encrypted USB stick, and the two go in a safe; the local copy gets shredded. The how-to even has you type the paper backup back in once before you walk away, because discovering your printer ate a stripe of pixels is a problem you want now, not eighteen months from now when the building’s on fire.

The public half, though, has been in your trust set the whole time, embedded in the binary and served over WKD right alongside the signing keys. That’s the trick. Because every installed client already trusts the rotation authority, it can vouch for a brand-new signing key outside the normal sign-with-the-old-key path. You bring the private half out of the safe, use it to authorise the new key, ship that, and installed tools adopt the replacement on their next update, all without the dead primary key ever having to sign a thing.

It is, deliberately, a key you hope never to touch. But a break-glass key you forgot to cut is just a pane of glass.

The whole chain, end to end

That’s the series. Step back and the shape is one clean line: a signing key is born inside KMS and never leaves it (Part 2); its public half is minted out of the HSM without the private bytes ever surfacing (Part 4); that public key is published off-platform over WKD, somewhere your release host can’t quietly rewrite (Part 4), and embedded into the binary as a required trust anchor (Part 5); every tagged release is signed through the key over short-lived OIDC credentials with no stored secrets (Parts 3 and 6); a stranger’s copy of your tool verifies its own updates against that anchor before trusting a byte (the cross-check); and when the day comes, the whole thing is rotatable without locking anyone out. A key that can’t be stolen, can’t be forged, and can still be replaced.

If you’ve followed all seven parts, you’ve built that. If you’ve dipped in for one piece, the pillar ties the lot together and points at the deep-dives behind the why. Either way, your users are getting updates they can actually trust, which was the whole point. Go and leave your supply chain better than you found it.

Sign your own binaries with go-tool-base, part 6: sign every release with GoReleaser

Mon, 22 Jun 2026 00:00:00 +0000

By now you’ve got all the pieces lying on the bench. A KMS key that signs but never hands over its private half (Part 2). A CI role you can assume over OIDC with no stored credentials (Part 3). A public key you’ve minted and published (Part 4), and embedded in the binary (Part 5). What you don’t have yet is the bit that makes it routine: a release that signs itself, every time, without you remembering to do anything. That’s this part. We wire signing into the tagged-release pipeline so that pushing a v* tag is the whole ceremony.

This is the part of the signing series where the chain stops being a set of commands you run by hand and becomes something the pipeline does for you. We’re using GoReleaser, which already builds your binaries, writes a checksums file and cuts the release. It needs one extra trick: sign the checksums on the way out, through the KMS key, using credentials that only exist for the length of the job. And you don’t hand-write that trick into your release config. gtb does.

You’ll need a working GoReleaser setup releasing a go-tool-base CLI, plus the KMS key, the signer role and the embedded public key from the earlier parts.

One command wires the signing in

In Part 5 you ran gtb enable signing to turn on the verifying side: embed the key, check every update against it. Now you give that same command the key the release pipeline should sign with, and it wires the producing side too:

gtb enable signing --key-id alias/acme-release-signing-v1

That records the KMS key in your .gtb/manifest.yaml and regenerates .goreleaser.yaml with a signs: block that calls gtb sign. The WKD email and everything else you set in Part 5 stay exactly as they were; you’re adding the key, not starting over. Because the block is generated, you don’t hand-edit the release config any more than you hand-edit the embed wiring. Change the key later (a new region, a rotated alias) and you re-run the command, not the YAML.

The generated block looks like this:

signs:
 - id: checksums
 cmd: gtb
 args:
 - "--ci"
 - "sign"
 - "--backend"
 - "aws-kms"
 - "--kms-region"
 - "eu-west-2"
 - "--key-id"
 - "alias/acme-release-signing-v1"
 - "--public-key"
 - "internal/trustkeys/keys/signing-key-v1.asc"
 - "--output"
 - "${signature}"
 - "${artifact}"
 artifacts: checksum
 signature: "${artifact}.sig"
 output: true

The region defaults to eu-west-2 and the public key to the internal/trustkeys/keys/signing-key-v1.asc you embedded in Part 5; pass --kms-region or --public-key if yours differ. The backend defaults to aws-kms, which is the one that matters in CI.

Why only the checksums

artifacts: checksum is the line doing the load-bearing work. It tells GoReleaser to run the signing command once, over the checksums manifest only, not over every binary and archive in the release.

Here’s why that’s enough. GoReleaser builds your binaries, then writes a checksums.txt listing the SHA-256 of each one. Every artefact in the release is named in that file by its hash. So if you sign the manifest, you’ve transitively vouched for everything it lists: change a single byte of any binary and its hash no longer matches the line in checksums.txt, and the moment you alter checksums.txt to cover for that, the signature over it breaks. One signature, the entire release covered, through the hash chain. The per-binary build stays completely untouched, which keeps reproducible builds reproducible. signature: "${artifact}.sig" names the output checksums.txt.sig, the detached, ASCII-armored OpenPGP signature your tool looks for on every self-update.

No shim, just `gtb`

If you go reading go-tool-base’s own release config, you’ll find its signs: block points at a scripts/sign-release.sh shim rather than calling gtb directly. Yours doesn’t, and the difference is worth understanding.

go-tool-base is signing the very binary it’s in the middle of building, so it can’t use an installed gtb to do it. Its shim runs go run ./cmd/gtb to build a throwaway signer from source. It also reads the key id, public key and region from environment variables, because the one config has to serve every build. Your tool has neither problem. It calls the gtb you already installed, and the generator already knows your key id, region and public-key path because you just told it. So there’s nothing for a shim to abstract: the whole invocation goes straight into args:, where you can read it.

The one thing you do still need is gtb on the release runner’s PATH. Install it in a before_script, or bake it into your CI image, the same way you would any other release tool.

Where the credentials come from

gtb sign asks KMS to sign. KMS will only oblige if the caller has credentials, and the whole point of keyless CI is that there are no stored credentials to leak. So they’re minted on the fly.

On GitLab, the release job declares an id_tokens: block. GitLab injects a short-lived OIDC token (a JWT) into the job, the before_script writes it to a file, and the AWS SDK’s default credential chain picks it up from there. No aws CLI call, no assume-role-with-web-identity you write yourself: set AWS_ROLE_ARN and AWS_WEB_IDENTITY_TOKEN_FILE and the SDK does the web-identity exchange the first time gtb sign touches KMS:

goreleaser:
 id_tokens:
 AWS_WEB_IDENTITY_TOKEN:
 aud: sts.amazonaws.com
 variables:
 AWS_REGION: eu-west-2
 AWS_ROLE_ARN: arn:aws:iam::…:role/acme-release-signing-v1-signer
 AWS_WEB_IDENTITY_TOKEN_FILE: /tmp/oidc-token
 before_script:
 - echo "$AWS_WEB_IDENTITY_TOKEN" > "$AWS_WEB_IDENTITY_TOKEN_FILE"

The key id and region live in the generated signs: block now, not here, so CI only supplies the AWS credentials. AWS_REGION still earns its place: the --kms-region flag points the KMS client at the key, while the SDK uses AWS_REGION for the STS exchange that mints the credentials in the first place. The aud has to match the audience the signer role’s trust policy expects (Part 3 set this up; for the OIDC provider go-tool-base uses, that’s sts.amazonaws.com). On GitHub the moving parts are the same, you just let aws-actions/configure-aws-credentials do the token-to-credentials dance instead of writing the file yourself.

Those credentials are scoped tight. The signer role’s trust policy pins it to this project’s tag pipelines, so even if the role ARN leaked, nothing but a release tag on your repo can assume it.

Don’t sign when there’s nothing to sign with

A local goreleaser release --snapshot, or a CI run that isn’t a release, has no OIDC token and no business reaching for KMS. GoReleaser is told to skip the whole signing step in that case: the release job runs with --skip=sign unless the web-identity token is present. So a non-release build never so much as looks at KMS, which is exactly what you want when you’re iterating on your laptop. The signing only fires on the real thing.

Cut a release

With all of that in place, releasing is one push:

git tag v1.4.0
git push origin v1.4.0

The tag pipeline fires. GoReleaser builds the binaries, writes checksums.txt, calls gtb sign, gtb asks KMS to sign over OIDC, and checksums.txt.sig lands next to the manifest. Both get attached to the release. Nobody typed a signing command and no private key was anywhere near the runner.

A two-person gate, if you want one

There’s a window worth thinking about: a compromised CI runner during a release could, in principle, ride the OIDC credentials to get one malicious thing signed. You can shut that window with an approval gate in front of the signing job. On GitLab, a protected environment with a required approval (or a manual job) makes the release wait for a second pair of eyes; on GitHub, “required reviewers” on the release environment does the same. It’s optional, and it adds friction to every release, so weigh it against how exposed your runners are. For a lot of projects the OIDC scoping alone is enough; for anything where a forged release would be a genuine incident, the gate is cheap insurance.

Where this leaves you

Every release you cut from here on carries a signature made by a key you control and verifiable by anyone, and you got there by pushing a tag. That closes the loop the series has been building toward: the production side now actually produces signed releases.

Which means you can finally pull the trigger on the bit Part 5 left primed. We embedded the public key and left enforcement off, because turning it on before you ship signatures would brick every update. Now that signatures are shipping, Part 7 deals with the part everyone skips and nobody can afford to: rotating the key, and getting yourself out of trouble if one ever goes bad, without locking your users out.

Sign your own binaries with go-tool-base, part 4: mint and publish your public key

Sat, 20 Jun 2026 00:00:00 +0000

By the end of Part 3 your release pipeline can sign through a KMS key it never holds, over credentials that expire in minutes. The private half is locked away exactly where you want it. There’s a snag, though: a signature is no use to anyone who can’t get hold of the matching public key, and a KMS key won’t hand you one. KMS deals in raw signing operations, not OpenPGP entities. So this part does two things: it produces the published public key from the KMS key without ever touching the private bytes, then puts that key somewhere your release platform can’t reach.

That last bit is the part people skip, and it’s the part that does the real work. The whole scheme in a signature the platform can’t forge rests on the verifying key living somewhere an attacker can’t poison in the same breath as the release. We’ll come back to why that matters once the key is in hand.

Mint the public key from KMS

gtb keys mint builds an OpenPGP public key out of a signing backend. In Part 1 the backend was a .pem on disk; now it’s aws-kms, and only that flag changes:

gtb keys mint \
 --backend aws-kms \
 --kms-region eu-west-2 \
 --key-id alias/acme-release-signing-v1 \
 --name "Acme Releases" \
 --email release@acme.dev \
 --created "2026-06-02T00:00:00Z" \
 --output signing-key-v1.asc

INFO Minted OpenPGP key backend=aws-kms key_id=alias/acme-release-signing-v1 output=signing-key-v1.asc creation_time=2026-06-02T00:00:00Z fingerprint=...

Here’s the neat part, and it’s worth pausing on. An OpenPGP key isn’t just a lump of public-key material; the key carries a self-signature, a signature it makes over itself to bind the name and email to the key. So minting a public key normally needs the private key to do that signing. mint doesn’t. It wraps the backend in a crypto.Signer, and every signing operation, the self-signature included, becomes a kms:Sign call. KMS does the maths inside the HSM and hands back a signature; no private byte is ever exported, not even to stamp the key’s own identity onto itself. The mechanism is the subject of a signing key that never leaves KMS if you want to see how the signer is wired up.

Two things to get right. First, the AWS credentials: minting needs both kms:GetPublicKey and kms:Sign on the key, because it reads the public material and then signs the self-signature with it. The signer role you stood up in Parts 2 and 3 can do both; running this locally, your own credentials need the same. Second, and this is the same lesson Part 1 hammered on, pin --created. An OpenPGP fingerprint is derived partly from the creation time, so a different timestamp gives you a different fingerprint and, in effect, a different key as far as your tooling is concerned. Use the moment the KMS key was created and never let it drift: the key you embed in Part 5 and the key you publish here have to be byte-identical, and --created is what guarantees it.

Mint the rotation authority while you’re here

You won’t use it until Part 7, but the rotation-authority key is far easier to create now, alongside everything else, than to bolt on in a panic when you need it. It’s a break-glass key: an offline key whose only job is to vouch for a new signing key if the KMS one ever has to be replaced. The spare front-door key you tape to the back of a drawer, not the one on your keyring.

Because it’s break-glass, it isn’t a KMS key. You generate it on a trusted offline machine and the private half goes straight into cold storage:

gtb keys generate \
 --algorithm ed25519 \
 --name "Acme Rotation Authority" \
 --email release@acme.dev \
 --created "2026-06-02T00:00:00Z" \
 --output rotation-authority.asc

INFO Generated OpenPGP keypair algorithm=ed25519 public_output=rotation-authority.asc private_output=rotation-authority.priv.asc creation_time=2026-06-02T00:00:00Z fingerprint=...
WARN Move the private-half file to offline storage now. private_output=rotation-authority.priv.asc

For Ed25519 you get an armored public .asc and an armored secret-key block (.priv.asc, the same wire format gpg --export-secret-keys produces). Do what the warning says, and do it before you forget: move rotation-authority.priv.asc to offline storage immediately. An encrypted USB stick and a paper backup is not paranoid for a key you might not touch for two years and will desperately need when you do.

Notice it carries the same release@acme.dev user ID as the signing key, not a separate rotation@ address. That’s deliberate, and it’s the one place people trip: WKD groups keys by the email in their UID, so the matching address is what puts the rotation authority in the same bucket as the signing key, and into the same embedded trust set. Give it a different email and it quietly drops out of the published bucket, your embedded and WKD key sets stop matching, and the cross-check from Part 5 starts failing every update. Same release identity, two keys. The public half travels with the signing key from here on, published and embedded together.

Build the WKD tree

Now publish them. The way clients find a public key from an email address is the Web Key Directory: a fixed set of files under .well-known/openpgpkey/ on a web server, where each key lives at a path derived by hashing the local-part of its email. gtb keys wkd builds that tree for you, in pure Go, with no gpg or gpg-wks-client anywhere in sight:

gtb keys wkd \
 --domain acme.dev \
 --email release@acme.dev \
 --output ./wkd-staging \
 signing-key-v1.asc rotation-authority.asc

INFO WKD bucket email=release@acme.dev hash=... keys=2
INFO wrote path=wkd-staging/.well-known/openpgpkey/acme.dev/policy
INFO wrote path=wkd-staging/.well-known/openpgpkey/acme.dev/submission-address
INFO wrote path=wkd-staging/.well-known/openpgpkey/acme.dev/hu/...
INFO WKD tree complete output=wkd-staging method=advanced emails=1 files=3

Both keys carry release@acme.dev in their UID here, so they land in the same hu/ bucket, concatenated. (If you’d rather split them, give each a different email and pass --email twice.) The tree under ./wkd-staging holds a policy file (required by the spec, and empty), a submission-address file, and one hu/<z-base-32-hash> file per email with the keys inside.

One decision matters for where this gets served. The default --method is advanced, which serves the tree from a dedicated openpgpkey.acme.dev subdomain, which is why the path above has acme.dev nested inside it. Pass --method direct instead and the tree is served from acme.dev itself. Advanced is the modern default and what you want unless you’ve a reason otherwise; it does mean you’ll need DNS and TLS for openpgpkey.acme.dev.

Publish it somewhere the platform can’t reach

This is the bit that’s tempting to fudge, and the bit you mustn’t. Do not drop the WKD tree onto the same host, or under the same account, as your code and your releases.

Walk the attack through. Your binary will carry an embedded copy of the public key (Part 5). On every self-update it fetches the published key over WKD and checks the two agree before trusting a download. That cross-check is the entire defence. If an attacker who compromised your release platform could also rewrite the WKD tree, they’d swap both keys for one of their own, sign a malicious release with it, and the client would wave it straight through. The cross-check would be comparing a forged key against another copy of the same forged key. Worthless. The defence only holds if the published key and the embedded key come from infrastructure an attacker would have to breach separately. That argument is laid out in full in a signature the platform can’t forge; this is where you actually pay for it.

In practice that means a different host with its own credentials. A static host like Cloudflare Pages in Direct Upload mode does the job: you build the tree locally and push it with the Wrangler CLI under a token scoped to Pages edit and nothing else, no Git integration wired to your code repo. The token that can rewrite your keys has no power over your releases, and vice versa. For the advanced method you’ll also point DNS for openpgpkey.acme.dev at that host and let it terminate TLS. Whatever you pick, the test is simple: could one stolen credential change both the key in the binary’s update path and the key on the server? If yes, you’ve built a very elaborate way of trusting nobody.

Where this leaves you

You’ve turned a key locked inside KMS into a published, fetchable OpenPGP key without the private half ever surfacing, minted the offline rotation authority you’ll be glad of later, and put both somewhere your release platform can’t quietly rewrite. The verifying side of the loop now exists out in the world, ready to be checked against.

What’s missing is the checking. Part 5 bakes the public key into the binary as its trust anchor and turns enforcement on, so the tool refuses any update whose signature doesn’t hold, the moment that update lands.

Sign your own binaries with go-tool-base, part 3: keyless CI signing with OIDC

Fri, 19 Jun 2026 00:00:00 +0000

Part 2 left you with a KMS key your release pipeline can sign through and a role (<name>-signer) that’s allowed to call kms:Sign and nothing else. There’s one obvious question left hanging: how does a CI job become that role without an AWS access key stashed in a CI variable? Because a long-lived key sitting in your settings is exactly the kind of credential that ends up in a breach write-up. This part wires CI in with no stored credentials at all, on GitLab and GitHub.

The mechanism is OIDC federation, and the one-sentence version is: your CI platform already proves who it is to AWS, so let it. When a pipeline runs, GitLab or GitHub can hand the job a short-lived signed token (a JWT) that says “this is a tag pipeline on acme/acme-cli”. AWS trusts that token the same way a bouncer trusts a passport: it checks who issued it and what it says, and if the claims match a role’s trust policy, it swaps the token for temporary AWS credentials that live only for the job’s run. No key is stored anywhere; the credentials are minted on the spot and evaporate when the runner stops. The deep-dive on keyless CI covers the why and the threat model; here we just do the wiring.

Two things have to line up for this to work: the IAM OIDC identity provider (the thing in your AWS account that says “I trust tokens from this issuer”), and the signer role’s trust policy (which says “and only from these pipelines”). The terraform-aws-signing-kms module owns the second. The first comes from a sibling module.

Where the identity provider comes from

You don’t register the OIDC provider in the signing module. That’s deliberate: the same provider is shared across every role in the account that federates from CI, so it lives one level up, in terraform-aws-bootstrap. It takes a ci_provider input (github by default, or gitlab), registers the right IAM OIDC identity provider for that forge, and emits its ARN. You feed that ARN into the signing module. If you ran the bootstrap in Part 2 you already have it; if not, stand it up first and grab the oidc_provider_arn output.

That ARN is the only thing the two modules need to agree on. Everything else about which pipelines may sign lives in the signing module’s trust policy, which is what the rest of this part configures.

The GitLab path

Look the provider up by URL so you never hardcode the ARN, then pass it in along with the subject filter that scopes who can assume the role:

data "aws_iam_openid_connect_provider" "gitlab" {
 url = "https://gitlab.com"
}

module "signing_kms" {
 source = "gitlab.com/phpboyscout/signing-kms/aws"
 version = "0.1.2"

 name = "acme-release-signing-v1"
 oidc_provider_arn = data.aws_iam_openid_connect_provider.gitlab.arn

 # oidc_issuer_host defaults to "gitlab.com"
 # oidc_audience defaults to "sts.amazonaws.com"

 ci_subject_filters = [
 "project_path:acme/acme-cli:ref_type:tag:ref:v*",
 ]

 key_administrator_arns = [/* ... from Part 2 ... */]
 automation_role_arn = data.aws_iam_role.automation.arn
}

The interesting line is ci_subject_filters. GitLab stamps each CI token’s sub claim with the project path, the ref type and the ref. The pattern above reads as “tag pipelines on acme/acme-cli, for any ref starting v”. A branch pipeline or a merge-request pipeline carries ref_type:branch instead, so it simply doesn’t match, and the role refuses to be assumed. Your signer can only be driven from a release tag, which is the whole point: a dependency author opening an MR can’t trick CI into minting a signature.

On the pipeline side, the release job declares an id_tokens block so GitLab issues a token with the right audience, writes it to a file, and the AWS SDK picks it up:

goreleaser:
 rules:
 - if: '$CI_COMMIT_TAG =~ /^v\d+\.\d+\.\d+$/'
 id_tokens:
 AWS_WEB_IDENTITY_TOKEN:
 aud: sts.amazonaws.com
 variables:
 AWS_REGION: eu-west-2
 AWS_ROLE_ARN: ${SIGNER_ROLE_ARN}
 AWS_WEB_IDENTITY_TOKEN_FILE: /tmp/aws-token
 before_script:
 - echo "$AWS_WEB_IDENTITY_TOKEN" > /tmp/aws-token
 script:
 - aws sts get-caller-identity
 - goreleaser release --clean

AWS_ROLE_ARN plus AWS_WEB_IDENTITY_TOKEN_FILE is the convention the SDK recognises: it sees the two together and calls assume-role-with-web-identity for you, so by the time goreleaser runs it’s already the signer role. The actual signing job is Part 6; the aws sts get-caller-identity line is just a sanity check that federation worked. It should print the signer role’s ARN.

The GitHub path

Same shape, different issuer and a different sub format. GitHub’s sub support landed in module v0.1.2: earlier versions validated ci_subject_filters against GitLab’s format only and would reject a GitHub subject outright, so pin the version.

data "aws_iam_openid_connect_provider" "github" {
 url = "https://token.actions.githubusercontent.com"
}

module "signing_kms" {
 source = "gitlab.com/phpboyscout/signing-kms/aws"
 version = "0.1.2"

 name = "acme-release-signing-v1"
 oidc_provider_arn = data.aws_iam_openid_connect_provider.github.arn
 oidc_issuer_host = "token.actions.githubusercontent.com"

 # oidc_audience still defaults to "sts.amazonaws.com"

 ci_subject_filters = [
 "repo:acme/acme-cli:ref:refs/tags/v*",
 ]

 key_administrator_arns = [/* ... */]
 automation_role_arn = data.aws_iam_role.automation.arn
}

Two differences from GitLab. The oidc_issuer_host has to change, because it’s the prefix on the trust-policy condition keys (token.actions.githubusercontent.com:sub rather than gitlab.com:sub). And the subject format is GitHub’s own: repo:<owner>/<repo>:ref:refs/tags/v* scopes the same way the GitLab pattern did, to tag refs only. The audience stays sts.amazonaws.com, because that’s the default aws-actions/configure-aws-credentials requests, so there’s nothing to override.

The workflow side is the official AWS action. It needs id-token: write permission to ask GitHub for the token in the first place, and contents: write so GoReleaser can create the release:

permissions:
 id-token: write
 contents: write
jobs:
 release:
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v4
 - uses: aws-actions/configure-aws-credentials@v4
 with:
 role-to-assume: ${{ secrets.SIGNER_ROLE_ARN }}
 aws-region: eu-west-2
 audience: sts.amazonaws.com
 - run: aws sts get-caller-identity
 - run: goreleaser release --clean

If you forget id-token: write, GitHub never issues a token and the action fails before it reaches AWS. It’s the most common GitHub-side trip-up.

The gotcha that wastes an afternoon: token audience

Here’s the one worth burning into memory, because the error message points you at the wrong thing. An IAM OIDC provider carries a client_id_list, and it rejects any token whose aud claim isn’t on that list before AWS even looks at the role’s trust policy. So if your CI token’s audience and the provider’s client ID don’t match, you get:

InvalidIdentityToken: Incorrect token audience

That reads like a role-trust problem, and you’ll go round in circles editing ci_subject_filters, but the trust policy was never consulted. The fix is to keep the audience aligned to sts.amazonaws.com everywhere: it’s the module’s oidc_audience default, it’s what terraform-aws-bootstrap puts on the provider’s client_id_list, it’s the aud in the GitLab id_tokens block, and it’s the default aws-actions/configure-aws-credentials requests. Leave all four alone and they agree by default. The moment you override one, override all of them, or you’ll meet that error.

Where this leaves you

There’s now no AWS access key anywhere in either forge. The pipeline proves it’s a release tag on your project, AWS hands it the signer role for the length of the run, and the credentials are gone the moment the runner stops. The signing module’s trust policy is the gate; the bootstrap module’s OIDC provider is the lock it hangs on. Both are verifiable in the public modules: terraform-aws-signing-kms for the role and terraform-aws-bootstrap for the provider.

The role can sign, but you still can’t verify anything yet, because nobody has the public half of that KMS key. Part 4 fixes that: gtb keys mint pulls the public key straight out of KMS, and gtb keys wkd publishes it somewhere the release platform can’t touch.

Sign your own binaries with go-tool-base, part 2: a signing key in AWS KMS

Wed, 17 Jun 2026 00:00:00 +0000

Part 1 left you with a working signing loop and one glaring weakness: the private key was a .pem on your laptop, and files get copied. This part fixes that. You’ll generate the production signing key inside AWS KMS, where it’s created and never comes out, and stand up a role that can sign with it. The key itself is the only thing we build here; wiring CI in over OIDC is Part 3.

The big idea is the same one the deep-dive spends its whole length on, so I’ll keep it short: you never hold the private key and you never sign with it. You ask KMS to sign on your behalf with kms:Sign, and the private half stays inside the HSM for its entire life. There is no export, no download, no “just this once” copy onto a runner. An attacker who owns your CI still can’t walk away with the key, because the key was never on the runner to begin with.

What you’ll need first

This part is OpenTofu (or Terraform, the module is fine with either). You’ll need:

An AWS account you can apply infrastructure into.
An IAM OIDC identity provider already registered in that account. KMS doesn’t need it, but the signer role we create trusts it, so it has to exist. If you haven’t got one, the sibling module terraform-aws-bootstrap provisions it (it’s the same family as the terraform-aws-security-baseline module). Its oidc_provider_arn output is exactly what we feed in below.

You don’t need a CLI, a pipeline or a public key yet. This is just the vault and the key that lives in it.

The module

The key, its alias, the signer role and the full key policy come from one public module, terraform-aws-signing-kms. Here’s the whole consumer block:

data "aws_iam_openid_connect_provider" "gitlab" {
 url = "https://gitlab.com"
}

module "signing_kms" {
 source = "gitlab.com/phpboyscout/signing-kms/aws"
 version = "0.1.2"

 name = "acme-release-signing-v1"
 description = "Acme release binary signing"

 oidc_provider_arn = data.aws_iam_openid_connect_provider.gitlab.arn
 ci_subject_filters = ["project_path:acme/acme-cli:ref_type:tag:ref:v*"]

 key_administrator_arns = [ /* operator role + account root */ ]
 automation_role_arn = data.aws_iam_role.automation.arn
}

# Carry these forward: Part 3 wires the signer role into CI, Part 4 mints
# the public key from the alias.
output "signer_role_arn" {
 value = module.signing_kms.signer_role_arn
}

output "signing_key_alias" {
 value = module.signing_kms.key_alias_name
}

A few of those values are doing more than they look, so let’s walk them.

name is acme-release-signing-v1, with the v1 on the end deliberately. The name derives the role (<name>-signer) and the alias (alias/<name>), and both of those want to outlive the key. When you rotate to a new key in Part 7 you’ll mint a -v2 and repoint things, so bake the version in now rather than wishing you had.

ci_subject_filters is in GitLab’s OIDC sub format here, and it’s the line that says which pipeline is allowed to assume the signer role: tag pipelines for any v* ref on acme/acme-cli, and nothing else (no branch builds, no merge requests). It’s the heart of Part 3, so I’ll leave the full explanation there. For now, know it’s not optional: an empty list would trust every token from the issuer, and the module refuses to let you do that.

key_administrator_arns and automation_role_arn are the two roles that manage the key, and the distinction between them matters. More on that in a moment.

tofu init
tofu apply

What you just built

A single asymmetric KMS key, plus the IAM scaffolding around it. The key is created with key_usage = SIGN_VERIFY and the spec RSA_4096, which is the module’s default and the one you want.

The obvious question, if you’ve signed things before, is why RSA-4096 and not Ed25519, which is smaller and faster. Two reasons, and neither is preference. The first is that AWS KMS simply doesn’t offer Ed25519 for asymmetric signing, so it’s off the table the moment you decide the key lives in KMS. The second is that OpenPGP, the format your signatures end up in, ties its packet encoding to the signing algorithm: the algorithm isn’t a detail you can swap underneath, it’s written into the bytes. RSA-4096 is the spec that satisfies both constraints, so it’s the secure default and you shouldn’t need to touch it.

Two more things to note about the key itself:

enable_key_rotation is off, and that’s intentional. AWS’s automatic yearly rotation only works on symmetric keys; an asymmetric SIGN_VERIFY key can’t be auto-rotated, because a new key would mean a new public half and every embedded trust anchor breaking at once. Rotation for signing keys is a deliberate, staged operation (mint a new key, publish it, repoint the alias), which is its own part later in the series.
The deletion window defaults to 30 days, the longest AWS allows. For a key this important, the longest possible “oops, undo” window is the safe choice.

The module also creates a stable alias, alias/acme-release-signing-v1. Always reference the key through its alias, never the raw key ID. The alias is what survives rotation: when v2 arrives, the alias gets repointed and everything calling through it keeps working.

Four principals, one key policy

This is the part worth slowing down for. A KMS key is governed by its key policy, and this module writes one policy that names four classes of principal, each with deliberately different reach:

The account root keeps kms:*. That’s AWS’s recommended break-glass: if every other path is locked out, the account owner can still recover.
The key administrators (key_administrator_arns, typically your operator role plus root) can administer the key, schedule its deletion, that sort of thing. They are not signers, and they can’t sign.
The automation role (automation_role_arn, the role your infra apply pipeline assumes) can manage the key as a Terraform resource, read it, tag it, even change its policy. What it deliberately cannot do is kms:Sign. Think about why: the role that applies your infrastructure runs on every change, so if owning that role let an attacker mint signatures, you’d have handed the whole point of the exercise back. Managing the key and using the key are two different powers, and only the signer gets the second one.
The signer role can call kms:Sign, kms:GetPublicKey and kms:DescribeKey, on this one key, and that’s the entire list. It can’t read other keys, can’t administer this one, can’t delete anything.

Here’s the detail that catches people out: the signer role has no attached IAM policy at all. Its permissions live entirely in the key policy. That’s not an oversight, it’s the design. One document, the key policy, is the single source of truth for who can do what to this key, so there’s no second place to check and no way for an attached role policy to drift out of sync with the key policy and quietly grant something nobody intended. If you want to know who can sign, you read one file.

The signer role is assumable via OIDC (those ci_subject_filters again), which is what lets a CI job step into it without any stored credential. That federation is Part 3’s whole job.

The outputs you’ll carry forward

The module hands back everything the later parts consume. The ones you’ll actually use:

key_id and key_arn, the bare ID and full ARN of the key.
key_alias_name and key_alias_arn, the stable alias. key_alias_name is the one you’ll pass to gtb as the KMS key reference, because it survives rotation.
signer_role_arn and signer_role_name, the role CI assumes to sign. signer_role_arn becomes an environment variable in your pipeline next part.

Keep these to hand. Part 3 needs signer_role_arn; Part 4 needs key_alias_name to mint the public key out of KMS.

What this costs

Worth a quick word, because this is the one part of the series that puts a line on an AWS bill. An asymmetric KMS key runs about a dollar a month, plus a tiny per-signature charge on the kms:Sign calls. For release signing, where you sign a handful of checksums files a month, the per-signature cost rounds to nothing. A dollar a month for a key that can’t be stolen off a laptop is the cheapest security control in this entire series.

Where this leaves you

The key exists, it lives somewhere it can never leave, and there’s a role that can sign with it (and a separate role that pointedly can’t). What there isn’t yet is any way for your release pipeline to become that role without a long-lived credential sitting in CI.

That’s the gap Part 3 closes: federating GitLab and GitHub into the signer role over OIDC, so a tagged release can assume it for the length of one job and nothing is stored anywhere. The ci_subject_filters line we glossed over here is where it starts.

A signing key that never leaves KMS

Thu, 11 Jun 2026 00:00:00 +0000

The last post in this series walked through how a tool verifies a release signature the platform can’t forge. That post had a loose end dangling off the back of it, and I knew it the whole time I was writing. Because a signature has to be produced by a private key… and a private signing key is the single worst thing in this entire story to lose. Steal it, and you sign malware that sails through every check I spent two posts building, signature and all. So where does that key live? The answer I landed on is the one this whole post is about: inside AWS KMS, and it never comes out.

The only key you can’t steal

Think about where a signing key normally ends up. A file on a build server. A secret in CI. A key on the release engineer’s laptop, “just for the release, I’ll delete it after”. Every one of those is a copy, and every copy is one more thing somebody can read, exfiltrate, or quietly clone while your back is turned. You can wrap them in passphrases and vaults and rotation policies all you like, and you’re still standing guard over a thing that exists in a place you don’t fully control.

The way out is almost annoyingly simple to state: the only key nobody can steal is the one that was never anywhere to be stolen from. So don’t hold the key at all. Let something else hold it, somewhere it has no export path, and ask that thing to sign for you.

That thing is AWS KMS. This is the infrastructure side of the question I opened the signing series with, finally answered with real Terraform.

A key that’s born in the box and stays there

The signing key is an asymmetric KMS key, and the module that provisions it is small enough to read in one sitting:

resource "aws_kms_key" "this" {
 description = var.description
 key_usage = "SIGN_VERIFY"
 customer_master_key_spec = var.key_spec # default RSA_4096

 # Asymmetric SIGN_VERIFY keys do not support KMS-managed rotation;
 # rotation is handled by minting a new key (alias = `<name>-v2`) and
 # publishing the v2 public key alongside the v1 key (dual-sign window).
 enable_key_rotation = false

 policy = data.aws_iam_policy_document.key_policy.json
}

The private half of that key is generated inside KMS and there is no API that hands it back to you. You don’t sign with it the way you’d sign with a file. You call kms:Sign: the bytes you want signed go up, a signature comes back down, and the key itself never moves. An attacker who completely owns my CI, my account, my laptop, can ask KMS to sign things for as long as their access lasts… but they can’t walk off with the key and keep signing forever. The blast radius is “while I’m compromised”, not “until I rotate a key I didn’t know had leaked three years ago”.

Why RSA-4096 and not the Ed25519 I’d normally reach for? Because KMS asymmetric signing doesn’t offer Ed25519, and OpenPGP’s packet format is tied to the algorithm that signed it, so the choice of key spec ripples all the way out to the signature on the wire. RSA-4096 is the strong option KMS does offer, so RSA-4096 is what the workflow is built around. A constraint of the box shaped the cryptography, not the other way round, and I’d rather say so than pretend I picked RSA on purpose.

Minting an OpenPGP key from a key you can’t hold

Here’s the part I find genuinely neat. OpenPGP wants a private key to self-sign its own public key when you generate it. And I don’t have a private key in any form I can hand to a library… it’s sitting in KMS, behind a door with no handle on my side. So how do you produce a valid OpenPGP public key at all?

go-tool-base leans on a small Go interface, crypto.Signer: anything that can return its public key and sign a digest. A KMS-backed signer satisfies it by turning each Sign call into a kms:Sign request. Then pkg/openpgpkey builds the OpenPGP entity around that signer:

func Entity(signer crypto.Signer, name, email string, creationTime time.Time) (*openpgp.Entity, error) {
	rsaPub, ok := signer.Public().(*rsa.PublicKey)
	// ...
	pubPkt := packet.NewRSAPublicKey(creationTime, rsaPub)
	// Construct the private-key packet directly (rather than
	// packet.NewSignerPrivateKey, which panics on opaque signers):
	// the crypto.Signer drives the actual signing, so a KMS-backed
	// signer works here.
	privPkt := &packet.PrivateKey{PublicKey: *pubPkt, PrivateKey: signer}
	// ...
}

Look at that PrivateKey packet. The field where OpenPGP expects the secret key material holds the crypto.Signer instead, which is to say, a remote handle to KMS. When the entity self-signs its public key, that self-signature is computed by KMS. gtb keys mint runs exactly this and writes out an ASCII-armored OpenPGP public key, and at no point did a single byte of private key material exist on the machine that minted it. The OpenPGP “private key” is a phone line to a vault, not a key.

That public key is what gets published off-platform over WKD and baked into the binary, the two trust anchors that post cross-checks.

Access without a human and without a standing key

A key that never leaves KMS is only as good as the rules about who may call kms:Sign. The signer role is deliberately narrow: it can call kms:Sign and kms:GetPublicKey on this one key and nothing else, and it is assumable only over OIDC from specific CI subjects, the same keyless federation the rest of the estate runs on. No human holds it. No long-lived access key sits in a CI variable waiting to leak. A release job federates in for its few minutes, signs, and the credentials evaporate with the runner.

So the chain of “who can sign a release” has no standing secret in it anywhere. Not a key file, not an access key, not a console user. Just a short-lived token, scoped to two API calls, on a key that can’t be exported.

The real cost: rotation is manual

This isn’t free, and the bit it taxes you on is rotation. KMS won’t auto-rotate an asymmetric SIGN_VERIFY key, which is why the module sets enable_key_rotation = false rather than leaving a default on. Rotating means minting a new key (a -v2 alias), publishing its public key alongside the old one, and running a dual-sign window long enough that clients have picked up the new anchor before you retire the old. It’s manual, it’s a runbook, and pretending otherwise would be the kind of thing this series exists to argue against. The trade I made was: a key with no exfiltration path, in exchange for rotation I have to do by hand. For a release-signing key, that’s the right side of the trade.

Why this is a command and not a script I hid

The origin of all this is a good deal less tidy than the result. I was working through the key-generation runbook, creating the offline rotation key with a gpg command I’d copied straight off my own page… and it just hung. No error, no prompt, just a cursor blinking while gpg waited on something it never bothered to mention.

My first instinct was the lazy one: drop the minting script into a scripts folder in my infra repo and never speak of it again. Then it nagged. That repo’s private, so the recipe would live somewhere nobody else could ever reach, and I’d already half-promised myself a tutorial walking people through this exact setup. So it shouldn’t sit in infra at all. It should be a gtb command, with a pluggable backend so anyone can swap my KMS for whatever provider they happen to run.

The deeper objection is the one that actually shaped it, though. I didn’t want to be shelling out to gpg by hand in the first place. gtb is a tool I hand to other people, and every time it drops to the shell for some gpg incantation, that’s an environment I’m asking the next person to reproduce, a dependency to install, a fiddly step to get subtly wrong, all before they can sign a single thing. The aim was to keep as much of this inside the box as I could: mint the key, build the WKD tree, produce the signature, all in pure Go, with no gpg on the path and no gpg-wks-client either.

So gtb keys mint pulls the public half out of your KMS key and frames it as OpenPGP, the trick from earlier; gtb keys wkd builds the tree ready to upload; and gtb sign produces the detached signature through that same remote round-trip. What comes out is an entirely ordinary OpenPGP signature gpg --verify is happy with, so you’re not locked into anything of mine. And none of it is just for me: build your tool on go-tool-base and the same handful of commands stands you up with this exact model, pointed at your own KMS. No cloud KMS to hand? There’s a local backend, a plain key on disk, to wire the whole thing together on your laptop first. These are commands for you, the person shipping the tool. Your users never run mytool keys mint… they just get updates that quietly check themselves, which was the whole idea two posts ago.

That setup deserves a walkthrough of its own, and it’ll get one. For now, the ergonomics were the point, not a nicety bolted on afterwards. The safest setup in the world is no use to anyone if it takes a PhD to stand up.

Where this leaves the whole story

Step back and the full loop is finally closed. The private key is born in KMS and never leaves it. Its public key is minted from it, with KMS computing its own self-signature. That public key is published off-platform and embedded in the binary. Releases are signed by KMS, reached only through short-lived OIDC federation. And the client verifies against the embedded and WKD keys cross-checked against each other. At no single point in that chain is there a thing an attacker can grab that lets them forge a release, and the most dangerous thing of all, the private key, has no theft path because it has no export path.

That’s the thread running through the whole signing series, from the very first checksum to here: the strongest control isn’t a better lock on the key. It’s arranging things so the key was never somewhere you could lose it. Nobody is coming to clean your supply chain, so the least you can do is leave it nothing worth stealing.

Aws-Kms on PHP Boy Scout

Sign your own binaries with go-tool-base, part 7: rotation and break-glass

Why there’s no auto-rotate button

Rotate by minting a new key, never by changing the old one

The dual-publish, dual-sign window

The break-glass key, for when there’s no handover

The whole chain, end to end

Sign your own binaries with go-tool-base, part 6: sign every release with GoReleaser

One command wires the signing in

Why only the checksums

No shim, just gtb

Where the credentials come from

Don’t sign when there’s nothing to sign with

Cut a release

A two-person gate, if you want one

Where this leaves you

Sign your own binaries with go-tool-base, part 4: mint and publish your public key

Mint the public key from KMS

Mint the rotation authority while you’re here

Build the WKD tree

Publish it somewhere the platform can’t reach

Where this leaves you

Sign your own binaries with go-tool-base, part 3: keyless CI signing with OIDC

Where the identity provider comes from

The GitLab path

The GitHub path

The gotcha that wastes an afternoon: token audience

Where this leaves you

Sign your own binaries with go-tool-base, part 2: a signing key in AWS KMS

What you’ll need first

The module

What you just built

Four principals, one key policy

The outputs you’ll carry forward

What this costs

Where this leaves you

A signing key that never leaves KMS

The only key you can’t steal

A key that’s born in the box and stays there

Minting an OpenPGP key from a key you can’t hold

Access without a human and without a standing key

The real cost: rotation is manual

Why this is a command and not a script I hid

Where this leaves the whole story

No shim, just `gtb`