Terraform on PHP Boy Scout

Wire up an AWS billing alarm before the surprise

Tue, 14 Jul 2026 00:00:00 +0000

Everybody who has run anything on AWS carries the same low-grade dread: the bill you don’t see coming. A cron that spins up an instance and forgets to kill it, a misconfigured NAT gateway billing you by the gigabyte, a test that leaves a fleet running over a bank holiday weekend. The fix is well known and takes ten minutes: set a budget, and have it shout at you when spend crosses a line.

The trap is subtler, and it’s the reason for this post. You can set that budget, wire it to a topic, see the notification sitting in the console exactly as you configured it, feel thoroughly responsible… and then hear nothing at all on the day spend sails past your line. A billing alarm that doesn’t fire is worse than no alarm, because you’ve stopped watching for the thing yourself. So here’s how to wire one up properly, and the one piece AWS will actively nudge you into getting wrong.

The easy part: the budget

A budget is one resource. Give it a monthly cost limit and a threshold to shout at:

resource "aws_budgets_budget" "monthly_cost" {
 name = "monthly-cost"
 budget_type = "COST"
 limit_amount = "50"
 limit_unit = "USD"
 time_unit = "MONTHLY"

 notification {
 comparison_operator = "GREATER_THAN"
 threshold = 80
 threshold_type = "PERCENTAGE"
 notification_type = "ACTUAL"
 subscriber_sns_topic_arns = [aws_sns_topic.alerts.arn]
 }
}

You can point a notification straight at an email address, and plenty of people do. I’d send it to an SNS topic instead, because a topic is a fan-out point: today it emails you, tomorrow it also pokes Slack, a Lambda, or your on-call rota, and you change none of the budget wiring to do it. If you’ve built anything security-conscious you probably already have such a topic. The public terraform-aws-security-baseline module, for instance, stands up an alerts topic wired to shout about GuardDuty findings and root-account activity. Hanging your cost alarm off the same wire, so everything that matters arrives in one place, is a tidy instinct.

It’s also exactly where the trap is waiting.

The hard part: encryption locks the alarm out

A well-built alerts topic is encrypted, and it should be. The security-baseline one uses a customer-managed KMS key, so nothing sensitive ever sits in plaintext on the topic. Encrypt the topic, point your budget at it, and nothing about the setup will look wrong. The budget is real, the notification is configured against the topic, terraform apply is happy. But the alarm won’t fire, and AWS is going to be spectacularly unhelpful about why.

Here’s the mechanism, because it’s worth understanding rather than just pasting the fix. Most things that feed an alerts topic arrive through EventBridge, which publishes on your behalf. A budget doesn’t. AWS Budgets publishes to SNS directly, as its own service, under the principal budgets.amazonaws.com. To put a message onto an encrypted topic, whoever’s publishing has to encrypt it first, and that means being allowed to use the topic’s KMS key. EventBridge was granted that. The budget service was not.

You can see the grant the topic already makes in the module’s key policy: EventBridge is named as a user of the key, allowed to generate a data key and decrypt so it can encrypt what it publishes (alerts/main.tf). There’s no equivalent line for budgets.amazonaws.com, because the module was built for security findings, not cost. So the budget goes to publish, can’t touch the key, and the alert never leaves the building.

And here’s the bit that turns a five-minute fix into a lost afternoon: AWS’s own guidance points you the wrong way. Try to attach a budget to an encrypted topic in the console and it stops you with “The SNS topic is encrypted. The SNS topic won’t work without additional permissions. Disable encryption on the topic.” Read that twice. Faced with an encrypted topic a budget can’t use, AWS’s headline suggestion is to turn the encryption off. Don’t. You don’t rip out a good security control to make a billing alert work. And the belated warning you might get when a budget can’t publish, an out-of-band note asking you to make sure budgets is on the topic’s list of allowed publishers, points straight back at the topic’s access policy. Neither message ever mentions the key. Every signal AWS hands you is aimed at the wrong thing, and the real fix is to grant the alert access to the key, exactly the way the module already did for EventBridge.

The two grants that fix it

The budget needs the same treatment EventBridge got, in two places. First, on the KMS key policy, name the budget service as a user of the key (AWS documents this exact requirement):

statement {
 sid = "AllowBudgetsPublishViaTopic"
 effect = "Allow"
 principals {
 type = "Service"
 identifiers = ["budgets.amazonaws.com"]
 }
 actions = [
 "kms:GenerateDataKey*",
 "kms:Decrypt",
 ]
 resources = ["*"]
}

Second, on the SNS topic policy, let the budget publish:

statement {
 sid = "AllowBudgetsPublish"
 effect = "Allow"
 principals {
 type = "Service"
 identifiers = ["budgets.amazonaws.com"]
 }
 actions = ["sns:Publish"]
 resources = [aws_sns_topic.alerts.arn]

 condition {
 test = "StringEquals"
 variable = "aws:SourceAccount"
 values = [data.aws_caller_identity.current.account_id]
 }
 condition {
 test = "ArnLike"
 variable = "aws:SourceArn"
 values = ["arn:aws:budgets::${data.aws_caller_identity.current.account_id}:*"]
 }
}

Those two aws:SourceAccount and aws:SourceArn conditions are AWS’s documented shape for the budget topic policy, and they’re worth keeping: they stop any other account’s budget from ever using your topic (the confused-deputy guard the baseline already applies to its EventBridge statements). The pair of grants above are otherwise twins of what the module writes for EventBridge, with one principal swapped. Add them and the message flows.

One thing that catches people out: this only works with a customer-managed key. If your topic uses the default AWS-managed aws/sns key, you can’t edit its policy to add a service principal, so a budget can never publish to it. That’s a large part of why the security-baseline module brings its own CMK, and it’s the difference between “encrypted” and “encrypted and actually usable by the things that need it”.

Watch it fire once

The most important step is the one people skip, and it’s the whole reason I wrote this down. I won’t believe an alarm works until I’ve watched it fire, and a billing alarm is no exception. Temporarily drop the threshold to something you’re certain you’ve already crossed this month, a single percent if you like, apply, and wait for the message to actually land in your inbox or your Slack. When it does, you know the whole chain is live: budget, topic policy, key policy, subscription. Then put the threshold back.

It takes five minutes and it’s the difference between an alarm and a decoration. The budget was always the easy 80%. The KMS key policy is the fiddly 20% that decides whether, on the day a runaway resource starts eating your account, the thing pages you or just watches.

Sign your own binaries with go-tool-base, part 2: a signing key in AWS KMS

Wed, 17 Jun 2026 00:00:00 +0000

Part 1 left you with a working signing loop and one glaring weakness: the private key was a .pem on your laptop, and files get copied. This part fixes that. You’ll generate the production signing key inside AWS KMS, where it’s created and never comes out, and stand up a role that can sign with it. The key itself is the only thing we build here; wiring CI in over OIDC is Part 3.

The big idea is the same one the deep-dive spends its whole length on, so I’ll keep it short: you never hold the private key and you never sign with it. You ask KMS to sign on your behalf with kms:Sign, and the private half stays inside the HSM for its entire life. There is no export, no download, no “just this once” copy onto a runner. An attacker who owns your CI still can’t walk away with the key, because the key was never on the runner to begin with.

What you’ll need first

This part is OpenTofu (or Terraform, the module is fine with either). You’ll need:

An AWS account you can apply infrastructure into.
An IAM OIDC identity provider already registered in that account. KMS doesn’t need it, but the signer role we create trusts it, so it has to exist. If you haven’t got one, the sibling module terraform-aws-bootstrap provisions it (it’s the same family as the terraform-aws-security-baseline module). Its oidc_provider_arn output is exactly what we feed in below.

You don’t need a CLI, a pipeline or a public key yet. This is just the vault and the key that lives in it.

The module

The key, its alias, the signer role and the full key policy come from one public module, terraform-aws-signing-kms. Here’s the whole consumer block:

data "aws_iam_openid_connect_provider" "gitlab" {
 url = "https://gitlab.com"
}

module "signing_kms" {
 source = "gitlab.com/phpboyscout/signing-kms/aws"
 version = "0.1.2"

 name = "acme-release-signing-v1"
 description = "Acme release binary signing"

 oidc_provider_arn = data.aws_iam_openid_connect_provider.gitlab.arn
 ci_subject_filters = ["project_path:acme/acme-cli:ref_type:tag:ref:v*"]

 key_administrator_arns = [ /* operator role + account root */ ]
 automation_role_arn = data.aws_iam_role.automation.arn
}

# Carry these forward: Part 3 wires the signer role into CI, Part 4 mints
# the public key from the alias.
output "signer_role_arn" {
 value = module.signing_kms.signer_role_arn
}

output "signing_key_alias" {
 value = module.signing_kms.key_alias_name
}

A few of those values are doing more than they look, so let’s walk them.

name is acme-release-signing-v1, with the v1 on the end deliberately. The name derives the role (<name>-signer) and the alias (alias/<name>), and both of those want to outlive the key. When you rotate to a new key in Part 7 you’ll mint a -v2 and repoint things, so bake the version in now rather than wishing you had.

ci_subject_filters is in GitLab’s OIDC sub format here, and it’s the line that says which pipeline is allowed to assume the signer role: tag pipelines for any v* ref on acme/acme-cli, and nothing else (no branch builds, no merge requests). It’s the heart of Part 3, so I’ll leave the full explanation there. For now, know it’s not optional: an empty list would trust every token from the issuer, and the module refuses to let you do that.

key_administrator_arns and automation_role_arn are the two roles that manage the key, and the distinction between them matters. More on that in a moment.

tofu init
tofu apply

What you just built

A single asymmetric KMS key, plus the IAM scaffolding around it. The key is created with key_usage = SIGN_VERIFY and the spec RSA_4096, which is the module’s default and the one you want.

The obvious question, if you’ve signed things before, is why RSA-4096 and not Ed25519, which is smaller and faster. Two reasons, and neither is preference. The first is that AWS KMS simply doesn’t offer Ed25519 for asymmetric signing, so it’s off the table the moment you decide the key lives in KMS. The second is that OpenPGP, the format your signatures end up in, ties its packet encoding to the signing algorithm: the algorithm isn’t a detail you can swap underneath, it’s written into the bytes. RSA-4096 is the spec that satisfies both constraints, so it’s the secure default and you shouldn’t need to touch it.

Two more things to note about the key itself:

enable_key_rotation is off, and that’s intentional. AWS’s automatic yearly rotation only works on symmetric keys; an asymmetric SIGN_VERIFY key can’t be auto-rotated, because a new key would mean a new public half and every embedded trust anchor breaking at once. Rotation for signing keys is a deliberate, staged operation (mint a new key, publish it, repoint the alias), which is its own part later in the series.
The deletion window defaults to 30 days, the longest AWS allows. For a key this important, the longest possible “oops, undo” window is the safe choice.

The module also creates a stable alias, alias/acme-release-signing-v1. Always reference the key through its alias, never the raw key ID. The alias is what survives rotation: when v2 arrives, the alias gets repointed and everything calling through it keeps working.

Four principals, one key policy

This is the part worth slowing down for. A KMS key is governed by its key policy, and this module writes one policy that names four classes of principal, each with deliberately different reach:

The account root keeps kms:*. That’s AWS’s recommended break-glass: if every other path is locked out, the account owner can still recover.
The key administrators (key_administrator_arns, typically your operator role plus root) can administer the key, schedule its deletion, that sort of thing. They are not signers, and they can’t sign.
The automation role (automation_role_arn, the role your infra apply pipeline assumes) can manage the key as a Terraform resource, read it, tag it, even change its policy. What it deliberately cannot do is kms:Sign. Think about why: the role that applies your infrastructure runs on every change, so if owning that role let an attacker mint signatures, you’d have handed the whole point of the exercise back. Managing the key and using the key are two different powers, and only the signer gets the second one.
The signer role can call kms:Sign, kms:GetPublicKey and kms:DescribeKey, on this one key, and that’s the entire list. It can’t read other keys, can’t administer this one, can’t delete anything.

Here’s the detail that catches people out: the signer role has no attached IAM policy at all. Its permissions live entirely in the key policy. That’s not an oversight, it’s the design. One document, the key policy, is the single source of truth for who can do what to this key, so there’s no second place to check and no way for an attached role policy to drift out of sync with the key policy and quietly grant something nobody intended. If you want to know who can sign, you read one file.

The signer role is assumable via OIDC (those ci_subject_filters again), which is what lets a CI job step into it without any stored credential. That federation is Part 3’s whole job.

The outputs you’ll carry forward

The module hands back everything the later parts consume. The ones you’ll actually use:

key_id and key_arn, the bare ID and full ARN of the key.
key_alias_name and key_alias_arn, the stable alias. key_alias_name is the one you’ll pass to gtb as the KMS key reference, because it survives rotation.
signer_role_arn and signer_role_name, the role CI assumes to sign. signer_role_arn becomes an environment variable in your pipeline next part.

Keep these to hand. Part 3 needs signer_role_arn; Part 4 needs key_alias_name to mint the public key out of KMS.

What this costs

Worth a quick word, because this is the one part of the series that puts a line on an AWS bill. An asymmetric KMS key runs about a dollar a month, plus a tiny per-signature charge on the kms:Sign calls. For release signing, where you sign a handful of checksums files a month, the per-signature cost rounds to nothing. A dollar a month for a key that can’t be stolen off a laptop is the cheapest security control in this entire series.

Where this leaves you

The key exists, it lives somewhere it can never leave, and there’s a role that can sign with it (and a separate role that pointedly can’t). What there isn’t yet is any way for your release pipeline to become that role without a long-lived credential sitting in CI.

That’s the gap Part 3 closes: federating GitLab and GitHub into the signer role over OIDC, so a tagged release can assume it for the length of one job and nothing is stored anywhere. The ci_subject_filters line we glossed over here is where it starts.

One graph, not micro-stacks

Sun, 17 May 2026 00:00:00 +0000

Once an infrastructure repo has a few concerns in it (account hardening, the security baseline, the signing stack still to come) there’s a steady pressure to split them into separate stacks with separate state, and Terragrunt is right there to help you do it. The infra repo keeps everything in one OpenTofu graph instead. The reason comes down to who enforces your dependency ordering: the engine, or you.

The pressure to split

The infra repo’s src/ has several concerns in it, and more coming, the signing stack among them. Once a repo reaches that point, there’s a steady pressure to split: one stack per concern, each with its own state file.

It’s an appealing pressure. Separate stacks feel modular. Each apply touches less, so the blast radius of any one run is smaller. And Terragrunt exists, popular and well-regarded, precisely to orchestrate a fleet of separate stacks. The path is well trodden.

infra didn’t take it. src/ is a single OpenTofu root stack: each concern is a module block, in its own main.<concern>.tf file, all sharing one state and one graph.

What one graph gives you

The thing a single graph gives you is engine-enforced truth about ordering and data.

Inside one OpenTofu graph, the tool builds the full dependency DAG itself. When the signing stack needs a value the security baseline produced, you reference it directly, module.baseline.something, and OpenTofu guarantees two things: the baseline is created before the thing that depends on it, and the value handed across is the current one from this same apply. Ordering and data-passing aren’t things you arranged. They’re facts the engine checks and enforces, every plan, every apply.

What splitting costs

Split src/ into per-concern stacks with separate state, and that guarantee is the thing you spend.

Now one stack reads another’s outputs through terraform_remote_state. That’s a lookup of a snapshot: the other stack’s last applied state, whatever it was, whenever that was. It’s not a live edge in a graph. Ordering is no longer enforced by the engine either; it becomes something you arrange yourself, in CI stage sequencing or in Terragrunt’s own dependency blocks.

That’s the trade, stated plainly. You give up a strong, engine-checked guarantee, and you buy back a weaker, hand-arranged imitation of it. Terragrunt is a good tool for managing that weaker world tidily. But the question worth asking first is whether you should be in the weaker world at all.

When splitting is genuinely right

This isn’t an argument that splitting is always wrong. Separate states genuinely earn their place when concerns have different change cadences, different access boundaries, or different teams owning them: when you actively want an apply of one to be unable to touch another, and you want different people holding different state.

infra has none of those. It’s a single account, a single operator, one cohesive set of concerns. The only thing splitting would buy here is a smaller per-apply blast radius, and that’s better handled by reviewing the plan before it applies, which the next post is about, than by fragmenting the dependency graph. So src/ stays one graph, and Terragrunt was considered and deliberately not adopted.

If ordering between graphs is ever needed

If infra ever does genuinely need more than one stack, the plan isn’t Terragrunt. It’s to keep each stack a single strong graph internally, and to sequence the stacks with CI stages. Keep the engine-enforced guarantee where it’s strongest, inside each graph, and reach for hand-arranged ordering only at the one seam where it’s unavoidable.

Boiling it down

A multi-concern infrastructure repo feels like it should be split into per-concern stacks, and Terragrunt is right there to manage the result. infra keeps src/ as one OpenTofu graph instead.

Inside one graph, OpenTofu enforces dependency ordering and passes current values across module boundaries as checked facts. Split into separate states and that becomes a terraform_remote_state snapshot lookup plus ordering you arrange by hand: a weaker version of what you gave up. Splitting is right when concerns have different cadences, boundaries or owners; for a single-account, single-operator repo none of that applies, so the strong guarantee is worth keeping, and Terragrunt is the tool for a problem infra chose not to have.

Why I hand-rolled every module

Sun, 10 May 2026 00:00:00 +0000

There are well-known community module libraries for AWS: Cloud Posse, the terraform-aws-modules collection, plenty more. Both terraform-aws-bootstrap and terraform-aws-security-baseline use almost none of them. Every sub-module is hand-rolled from raw AWS resources, and before you accuse me of not-invented-here syndrome (a perfectly fair first guess), hear me out, because the same evaluation kept landing the same way for a real reason.

The promise of a wrapper module

The community module ecosystem makes an appealing offer. Don’t write raw aws_s3_bucket and aws_s3_bucket_policy and aws_s3_bucket_public_access_block and the rest. Call a tested, popular module, pass it a handful of inputs, and get a correct, well-configured bucket. Less code in your repo, and the code you don’t write has been exercised by thousands of other users.

For a lot of infrastructure that’s a genuinely good deal, and I take it often. For the two infrastructure modules in this series, I took it almost never. Every sub-module is built from raw AWS resources. That wasn’t a reflex. It was the same evaluation, made over and over, landing the same way.

What kept going wrong

For each place a wrapper module could have fitted, I looked at the wrapper. And the recurring finding was one of two things. Either using the wrapper correctly, with all the overrides my posture needed, came to more configuration than the raw resources would have. Or the wrapper’s abstraction leaked the instant I needed something it hadn’t anticipated, and I was now writing code to fight it.

The CloudTrail bucket, concretely

The clearest example is the bucket that holds CloudTrail logs.

There are popular modules that set up CloudTrail and bundle an S3 bucket for the logs. Convenient. But that bundled bucket isn’t the bucket I want. It doesn’t carry lifecycle { prevent_destroy = true }, and its bucket policy is weaker than the one the state bucket taught me to want: TLS-only, SSE-KMS-only, wrong-key-denied.

So to use the wrapper I had two options. Accept a weaker audit-log bucket than the rest of the account, which rather defeats the point of an audit log. Or fight the wrapper: disable its bucket, create my own, wire it back in. Fighting the wrapper is more work than simply writing the fifty-odd lines of raw aws_s3_bucket plus policy that give me exactly the posture I’d already designed once. The wrapper didn’t save code. It added a negotiation.

A wrapper is a deal, and deals have terms

This isn’t an argument that community modules are bad. It’s an argument about when the deal is good.

A wrapper module is a good deal while its abstraction holds: while what it assumes you want matches what you want. The moment you need something it didn’t anticipate, the deal inverts. Now you’re working against the abstraction, and an abstraction you’re fighting costs more than no abstraction at all. (Regular readers will recognise that line from the LangChain argument; it’s the same principle in a very different language.)

Infrastructure that holds signing keys is precisely the case where you need to control the specifics: every encryption setting, every lifecycle rule, every line of every bucket policy. That’s a domain where wrapper abstractions leak fast, because the whole job is the details the wrapper smoothed over.

The cost, paid on purpose

Hand-rolling isn’t free. It’s more lines of HCL in the repo, up front, than a one-line module call.

What those lines buy is worth the price for this kind of infrastructure. There’s no transitive module-version churn to track. There’s no abstraction between me and the resource when something behaves oddly. And every line is one I can read, and defend, in a security review, because I wrote it and it says exactly what it does. For a foundation that will hold the most sensitive key in the system, “readable and mine” beats “short and someone else’s”.

That’s a deliberate trade, not a universal rule. For an internal tool on a deadline, reach for the wrapper. For the security-critical base of everything else, the raw resources won every time I checked.

To sum up

The community module ecosystem offers less code that more people have tested, and for plenty of infrastructure that’s the right call. For terraform-aws-bootstrap and terraform-aws-security-baseline it almost never was, because each wrapper turned out to be more configuration than the raw resources once my posture was accounted for, or it leaked the moment I needed a specific.

The CloudTrail log bucket is the pattern in miniature: the bundled bucket lacked prevent_destroy and a strong policy, so using the wrapper meant either a weaker bucket or fighting the module. A wrapper is a good deal while its abstraction holds and a bad one the moment you fight it, and security-critical foundation infrastructure is all specifics. Hand-rolling cost more lines and bought code I can read and defend. For this, that was the trade worth making.

The chicken-and-egg of remote state

Wed, 06 May 2026 00:00:00 +0000

Here’s a puzzle that every infrastructure-as-code setup hits exactly once, right at the very beginning, and then never again. An OpenTofu stack stores its state in a backend. The bootstrap stack I wrote about last time has a particular job, and part of that job is to create the backend that remote state lives in. So where does the bootstrap stack store its own state, on the very first run, before it’s built the place state is supposed to go?

Where does the state of the thing that makes the state store live?

That’s the puzzle, and it’s a real ordering deadlock rather than a riddle.

An OpenTofu stack keeps a state file, and for anything shared that state file lives in a remote backend: on AWS, an S3 bucket. Fine. But the bootstrap stack has a particular job, and part of that job is to create the S3 bucket that remote state lives in.

So walk through the first run. Bootstrap has never been applied. The state bucket doesn’t exist, because creating it is what bootstrap is for. Bootstrap needs somewhere to store its own state. The only place that would make sense is the bucket it’s about to create, which isn’t there yet. The thing that builds the state store can’t store its state in the state store.

Run local, then migrate

The way out is a two-step that OpenTofu supports directly.

Bootstrap starts configured with a local backend: backend "local" {}. State is just a file on the operator’s machine. With that in place, the first tofu apply runs. It creates the S3 bucket and the KMS key, and records all of it in the local state file.

Now the bucket exists. So the backend configuration is rewritten to point at it: an s3 backend block naming the new bucket. Then tofu init -migrate-state. OpenTofu sees the backend has changed, picks up the local state file, and copies it into the S3 bucket. From that point on, bootstrap’s own state lives in the bucket that bootstrap created. The egg has laid the chicken.

The local backend was a scaffold. It existed for exactly one apply, to break the ordering deadlock, and then the state moved off it and it was never used again.

It happened twice

The infra repo actually did this migration twice, and the second time is the proof that the pattern is general rather than a one-off trick.

The first migration was the one above: local to S3, at the very start. The second came later, during the move from GitHub to GitLab. GitLab offers a managed HTTP state backend, and infra chose to use it. So the backend block was rewritten again, this time from s3 to http, and tofu init -migrate-state ran again, copying the state from the S3 bucket to GitLab’s backend.

The same move, twice, against three different backends. That’s the useful lesson hiding in the chicken-and-egg story. State is portable. The backend is just where you currently keep it, not a property of the stack itself, and moving it is a routine, supported operation rather than surgery.

Why this is the honest answer, not a hack

It’s easy to look at “apply once with a local backend, then migrate” and feel it’s a bit of a smell, a workaround for something that should have been cleaner.

It isn’t. It’s the honest answer to a real ordering problem, and the alternatives are worse.

The obvious alternative is to create the state bucket by hand, in the console, before running bootstrap at all. But then the most important bucket in the account is unmanaged. It exists outside every OpenTofu graph, nobody’s code describes it, its encryption and policy and prevent_destroy are whatever someone clicked that day, and it drifts. The local-then-migrate dance avoids exactly that. The bucket is created by bootstrap, described in code, and tracked in bootstrap’s own state from its very first apply. It’s managed from birth.

The chicken-and-egg isn’t a flaw to be embarrassed about. It’s just the shape of the problem when a stack has to build its own foundations, and OpenTofu’s -migrate-state is the supported tool for exactly that shape.

Pulling it together

Every OpenTofu stack needs a backend to store state, and the bootstrap stack’s job is to create the backend, so on its first run the bucket it needs doesn’t yet exist.

The resolution is to run bootstrap once with a local backend, let that apply create the bucket and key, then rewrite the backend configuration and tofu init -migrate-state the state into the bucket bootstrap just made. The infra repo did it twice, local to S3 and later S3 to GitLab, which shows the real point: state is portable, and the backend is just where you keep it. Doing it this way, rather than hand-creating the bucket, is what keeps that critical bucket managed in code from its very first day.

The bootstrap that does almost nothing

Fri, 01 May 2026 00:00:00 +0000

A brand-new AWS account is a slightly nerve-wracking thing. It can do almost anything, it’s hardened against almost nothing, and the list of stuff you ought to set up before you trust it with anything real is long. The natural instinct is to write one big “set up the account” module that does the whole list in a single apply. I want to talk you out of that, because the bootstrap module I’m happiest with does almost nothing, on purpose.

The first-apply problem

A brand-new AWS account is not ready for anything serious. Before you’d responsibly run real infrastructure into it, you want an account baseline: a password policy, account-wide S3 public-access blocking, default EBS encryption, CloudTrail, AWS Config, GuardDuty, alerting, a sensible human operator role. It’s a long list, and all of it matters.

The instinct, faced with that list, is to write one big “set up the account” module and have it do everything. One tofu apply, a fully prepared account, done.

That instinct is worth resisting, and terraform-aws-bootstrap resists it deliberately.

Three things, and a hard line

terraform-aws-bootstrap does three things:

state-backend, an S3 bucket and a customer-managed KMS key to hold remote Terraform state.
automation-iam, an OIDC identity provider and an IAM role that CI assumes to apply everything else.
nuke-config, which renders an aws-nuke configuration scoped to the account, for tearing a throwaway account back down.

That’s the whole module. Account hardening, CloudTrail, AWS Config, GuardDuty, the operator role, the alerting: none of it is in here. And it’s not absent by accident. The README has a section headed “what’s deliberately NOT in scope” that lists those exclusions out loud. The boundary is written down, because the boundary is the design.

Why the line is exactly there

The reason the line sits where it does is the most useful idea in the module.

Everything bootstrap excludes belongs in a separate stack, applied through the automation role bootstrap creates. Bootstrap’s only job is to get the account to the point where the next tofu apply can run properly: somewhere to store state, and an identity to run as. Once those two things exist, hardening the account isn’t a special bootstrapping act. It’s just another apply, done the normal way: in CI, reviewed, versioned, deployed through the role.

So the account baseline doesn’t need to be bundled into the bootstrap. It needs to be downstream of it. Bootstrap builds the on-ramp; it doesn’t also have to be the motorway.

A narrow module stays re-runnable

There’s a practical payoff to the narrowness, and it’s about fear.

Bootstrap is the one stack that can’t be applied through CI, because it’s what creates the CI identity in the first place. It runs locally, by a human, rarely. That’s exactly the kind of operation you want to be small, boring, and safe to repeat.

A bootstrap module that also did account hardening would be a large, stateful thing managing dozens of resources. Re-running it would be a held-breath operation. Keeping it to three concerns keeps it the opposite: a small stack you can read top to bottom, re-run without anxiety, and reason about completely. The narrowness isn’t minimalism for its own sake. It’s what keeps the one human-applied stack trustworthy.

The boundary is the feature

It’s tempting to judge a module by how much it does. A bootstrap module is the case where that’s exactly backwards. Its value is in how cleanly it stops.

terraform-aws-bootstrap does the bare minimum to make an account ready for the next apply, writes down everything it refuses to do, and hands off to a downstream stack for all of it. The next post follows the trickiest of its three jobs: the state backend has a genuine chicken-and-egg problem, because it has to store Terraform state in a bucket Terraform hasn’t created yet.

Where this leaves us

A fresh AWS account needs a long list of things before it’s safe, and the obvious move is one big module that does the lot. terraform-aws-bootstrap deliberately does only three: a state backend, a CI identity, and an account-scrub config. Everything else is written down as out of scope.

The boundary is the design. The excluded work belongs in a downstream stack applied through the CI role bootstrap creates, so hardening is just a normal reviewed apply rather than a bootstrapping special case. And keeping the one human-run, locally-applied stack small is what keeps it safe to re-run. A bootstrap module is judged by where it stops.