PHP Boy Scout — Media in pure Go

Introducing afmpeg and ffmpeg-wasi: FFmpeg with no install, no CGO, no disk

Sun, 28 Jun 2026 00:00:00 +0000

keryx renders short promo reels, and the way it does that, today, is the way nearly everything does: it shells out to the ffmpeg binary. Which is fine, until you ask it to render a project that doesn’t exist on disk. keryx can work on an in-memory project, a repo cloned straight into RAM with no local checkout, and the moment it tries to hand that to ffmpeg, the whole thing falls over. The binary wants real files in a real directory. There aren’t any.

I went looking for a way out and didn’t find one I could live with. The bindings that use purego/dlopen are immature and still need the host’s libav libraries installed. The CGO bindings to libav are mature and can absolutely work in memory, but they’re CGO, and CGO takes away the thing I most want from a Go program: a clean static cross-compile to a single binary that runs anywhere. The famous ffmpeg.wasm is an emscripten build aimed at the browser, which is the opposite target from a server-side Go tool. And the one existing WASI-capable build pins FFmpeg 5.1, which is end-of-life, and I’m not shipping an out-of-support media decoder whose entire job is parsing untrusted files. Every road had a tollbooth I wasn’t willing to pay.

So I built two new things instead, and because they’re so closely tied I’m introducing them together.

ffmpeg-wasi: current FFmpeg, sandboxed, CGO-free

ffmpeg-wasi (repo) is the foundation, and it’s the harder of the two to build. It takes FFmpeg’s media libraries, the libav* family, and builds them to wasm32-wasi, then drives them with a small purpose-built engine, producing a single .wasm artifact that runs anywhere a WASI runtime does. No native FFmpeg install, no C toolchain at deploy time, no shelling out to a binary. It’s built to run under wazero, the zero-dependency pure-Go WebAssembly runtime, so a Go program can transcode, filter and mux media embedded, sandboxed, and CGO-free, still cross-compiling to one static binary.

The interesting bit, the reason this didn’t already exist, is a wall that FFmpeg 7.0 put up. The 7.x series rewrote the command-line tool to be mandatorily multithreaded, and a pure-Go WASI runtime can’t run that, because the threading model it needs (wasi-threads, spawning real threads) isn’t something wazero does. Every project trying to get current FFmpeg into WASI hits that wall, which is exactly why the existing build froze at the last single-threaded CLI, 5.1, and went EOL there. ffmpeg-wasi goes under the wall instead of over it: it doesn’t compile the CLI at all. It links the libav* libraries directly, which build single-threaded without complaint, and drives them with its own engine. That’s the move nobody else has made, and it’s the whole reason this can track current, maintained FFmpeg rather than a frozen one.

afmpeg: the pure-Go binding that lives in memory

afmpeg (repo) was the catalyst for all this, and it’s the part a Go developer actually touches. It’s a small, idiomatic Go API (New, Run, Probe, Close) sitting on top of the ffmpeg-wasi artifact, with one important twist: its I/O is bridged to an afero.Fs. afero isn’t in the standard library, but it’s the filesystem abstraction a great deal of Go already reaches for when it wants to swap a real disk for something else, and that “something else” is exactly the point here. The inputs and outputs of a media job can live entirely in memory, or in any afero backend you like, and ffmpeg is none the wiser. keryx gets to render its in-memory project without ever touching the disk, which was the whole reason I started.

This isn’t a roadmap post. Both projects are released and run today: afmpeg is at v0.4.0, ffmpeg-wasi at n8.1.2-1 (current FFmpeg, not the EOL 5.1), and between them they do real in-memory transcodes, verified end to end, WAV to AAC and H.264 rescaled and re-encoded with x264. Stripped of keryx’s reel-specific filtergraph, the shape a caller actually deals with is about this small:

// Compile the ffmpeg-wasi module once, then reuse the runtime.
rt, err := afmpeg.New(ctx, afmpeg.WithModuleFile("ffmpeg-wasi-lgpl.wasm"))
if err != nil {
 return err
}
defer rt.Close(ctx)

// The whole job lives in memory: no temp dir, nothing on disk.
fs := afero.NewMemMapFs()
afero.WriteFile(fs, "in.wav", input, 0o644)

// Drive it with the ffmpeg arguments you already know...
res, err := rt.Run(ctx, fs, "-i", "in.wav", "-c:a", "aac", "out.m4a")
if err != nil || res.ExitCode != 0 {
 return fmt.Errorf("render failed: %w (%s)", err, res.Stderr)
}

// ...then read the finished file straight back out of memory.
out, _ := afero.ReadFile(fs, "out.m4a")

The .wasm itself is a published release artifact you pin by SHA-256 (or let afmpeg fetch and verify for you with WithModuleURL), so the licence boundary stays explicit and nothing surprising ends up in your binary.

It doesn’t do everything yet, and I’d rather say so than let you find out the hard way. Single input to single output and Probe work now; the full multi-pad filter_complex and multi-output muxing are the next thing on the bench. But the part that nobody had cracked, getting current FFmpeg to run sandboxed, pure-Go, over a virtual filesystem with nothing on disk, is done, and you can pull it today.

Why it’s two repos

There’s a reason these are separate projects rather than one, and it’s about licences, not tidiness. The moment you compile FFmpeg you’re handling LGPL and GPL code, and I wanted that boundary to be obvious rather than smeared across one repo where nobody’s quite sure what’s covered by what. So the build tooling and the (L)GPL .wasm artifacts live in ffmpeg-wasi, with no grey areas, and afmpeg stays a clean permissive layer on top of the published artifact. I’m also shipping both LGPL and GPL builds of the artifact, so anyone who just wants the output and doesn’t fancy doing their own FFmpeg build can pick the licence that suits them and get on with it.

Both repos are public, so you can rebuild or relink either one yourself, and the doc sites are now linked in the nav up top.

There’s a stack of stories behind these two already, the threading wall and the spike that found the way under it, the afero-to-WASI filesystem bridge, the day I built a reel-shaped API and then reverted it before merge for being too narrow, the licence decision in full. I’ll be writing those up as the work lands. For now this is the headline: I needed an FFmpeg I could embed in a Go program with no install, no CGO, and no disk, couldn’t buy one off the shelf, so I built it in the open, and as of today it’s there to pull.

The escape hatch that turned out to be french doors

Sat, 26 Sep 2026 00:00:00 +0000

This started because keryx needed to make videos, and I didn’t want to give it a disk.

That is honestly the whole of it. keryx builds reels, reels need FFmpeg, and I wanted an FFmpeg I could point at a virtual filesystem instead of the real one, something that reads and writes through an afero filesystem I handed it and never learns that a hard drive exists. A small ask, I thought, late one night at my desk… with a rather large problem hiding behind it.

FFmpeg is a CLI, and that’s the whole trouble

FFmpeg is magnificent and it is a command-line tool. So when an engineer needs it, they shell out to it, and two things follow from that (neither of them anyone’s fault).

The first is that you inherit FFmpeg’s arguments, which are a language of their own and not a friendly one. Most of us who have used it in anger have a shell script somewhere with a line in it we no longer understand and daren’t touch (I have several).

The second is the one I actually cared about. Shelling out means handing a process the run of the machine. It opens paths, seeks around, writes where it likes. If you’re processing something a stranger uploaded, that’s a decoder with your permissions and your filesystem, and “it’s fine, it’s just FFmpeg” is doing a lot of work in that sentence.

I’ve prided myself on being a security-first engineer for a long time, and that arrangement has always sat badly with me. What I also knew, and it’s the only reason I thought this was possible at all, is that you don’t have to talk to FFmpeg through a CLI.

Fifteen-odd years ago, as a PHP engineer, I used an extension that drove FFmpeg directly, with no shell and no arguments, just calls. So the shape existed already. Just not in Go.

Deliberately ignorant of the hardware

The answer was to compile FFmpeg to WebAssembly and run it as a guest, reading and writing through the afero filesystem I’d handed it. That became ffmpeg-wasi and afmpeg, and the filesystem trickery has its own post.

The important property is one it’s easy to mistake for a limitation: afmpeg has no idea what hardware it’s on. It doesn’t ask and it doesn’t care. Hand the same binary to a laptop, a CI runner or a container with nothing in it, and you get the same answer, because it never assumed a thing about the machine.

Not needing hardware is what makes it portable. That’s the design, and it stays the design. Where it got awkward was the day I wanted the hardware myself.

I was hedging even then

When the hardware-acceleration question first came up, I wrote the premise into a research prompt so it could be checked properly: Wasm implementations, in their current form, can’t reach hardware. And then I tacked “please correct me if I’m mistaken” onto the end of it.

Just as well. The tidy version of that claim is wrong. WebAssembly in a browser reaches the GPU perfectly well. That’s what WebGPU and WebGL are: host APIs the browser hands the module. The sandbox doesn’t grant that access, the host does, and that’s what a sandbox is for.

Outside a browser, same rule. The runtime has to offer it, no WASI standard obliges anybody to, and a runtime that wanted to would be binding to native graphics libraries to get there. afmpeg runs on wazero, which is zero-dependency and pure Go, and there’s a line in the justfile building the whole library with CGO_ENABLED=0 on every run so that promise can’t rot.

So the position was consistent. It had also, without me ever deciding it should, become a corner. There’s a difference between not requiring hardware and not permitting it, and afmpeg had the first only by having the second. So the plan was a small escape hatch: narrow, heavily guarded, and apologised for in the documentation.

Then I asked a much better question

The thing that changed it was noticing what I’d already built. afmpeg doesn’t consume anyone else’s FFmpeg. ffmpeg-wasi compiles its own, with its own driver interface, because making FFmpeg work under WASI meant building that interface anyway. It’s mine, and I ship it.

So could I layer an alternative interface onto the same thing, and distribute that variant too? The answer took an embarrassingly long time to land (weeks, if I’m honest, for something that now looks obvious). The interface built to make FFmpeg work in a sandbox is not sandbox-shaped at all. It’s just a clean way to drive FFmpeg without a command line, and if you point it at a native build instead of a Wasm one it works the same, minus the guest.

That is how spec 0028 stopped being about an escape hatch and started being about a second engine. The native driver runs as a separate process and is served the caller’s afero.Fs over a Unix socket, so the filesystem promise holds on both sides:

func (b *Backend) Invoke(ctx context.Context, fs afero.Fs, args ...string) (afmpeg.Result, error)

You still hand it a filesystem. It still can’t go looking for one. Swapping engines is one option, WithBackend, and the job API doesn’t change at all: same calls, byte-compatible results.

And it is significantly faster. I don’t mean marginally, or within noise. I mean the reel job stopped being a thing I started and then went to put the kettle on for, and became a thing that had finished before I’d stood up!

I’m deliberately not giving you a multiplier here, and there was one in an earlier draft of this. I’ve taken it out. The figures are being re-measured properly, on a quiet machine and against a couple of defects that were found in the meantime, and I suspect a single headline number is a poor way to describe what’s actually going on anyway. There’ll be a post with the full breakdown once the measurements are worth trusting.

For now: significantly faster, on the jobs keryx actually runs, which is where this all started. And because the native driver is a native binary, with real asm and real threads, the hardware is reachable from that side in a way it never was from the guest. I haven’t wired it up or put it through its paces yet, but it’s there, which is a good deal more than the corner was offering.

I’d intended a hatch.

What I’d actually built was a set of french doors… and I could throw them wide open!

The sentence I made myself write down

Running FFmpeg as a native subprocess is an objective security regression compared to WASM. I can’t spin that as a trade-off, or as “different security properties”. It’s worse.

The process boundary is real and the socket is a real constraint, but a native binary executing on your host with your permissions is not the same animal as a Wasm module that physically cannot make a syscall you didn’t hand it. That went into the documentation in those words, so anybody switching engines does it knowing what they’ve put down. A door you don’t have to answer for is a hole with a nicer name.

Still not the bottom rung

Having said all that, and meaning every word of it, there’s a comparison worth making. The alternative most people reach for isn’t WASM. It’s the shell.

Set the native driver against exec.Command("ffmpeg", ...), which is what the overwhelming majority of media pipelines actually do, and it comes out ahead on the thing that matters. Shelling out hands a process the run of the machine and a set of paths you typed. The native driver gets a socket, and on the other end of that socket is the afero filesystem afmpeg gave it: a filesystem you built, holding what you decided to put in it, with no directory to climb out of.

So the ladder runs like this. Shelling out at the bottom, with your whole disk in scope. The native driver in the middle, sandboxed at the filesystem even though the binary is native. WASM at the top, where the guest can’t reach anything at all without being handed it.

The middle rung is a step down from the top, and a long way up from where most of this work gets done. The default stays ignorant (runs anywhere, assumes nothing), and the hardware is a thing you open on purpose, and even then you don’t get the keys to the house.

Anyone can take it

ffmpeg-wasi is a standalone project. It isn’t a folder inside afmpeg and it doesn’t need afmpeg to be useful. It’s a custom FFmpeg build with a proper driver interface, distributed on its own, so any language with C or FFI bindings could drive the thing directly.

I’d be delighted if they used afmpeg as well, obviously. But they don’t have to, and that wasn’t the plan either. The plan was to stop keryx from having a disk.

Somewhere along the way it turned into an FFmpeg you can embed without shelling out, in a sandbox by default, at native speed when you ask, in whatever language you fancy. I still can’t tell you what half the FFmpeg arguments do!

FFmpeg thinks it has a disk

Mon, 24 Aug 2026 00:00:00 +0000

There’s a moment in every “FFmpeg in your own process” conversation where somebody asks where the files go. It’s a fair question. FFmpeg’s whole worldview is files: it opens paths, seeks around in them, writes headers and then goes back to fix them up. It assumes a disk is just there, the same way it assumes a clock is. Nobody ever wrote it to cope with either being a lie.

afmpeg doesn’t give it one. The FFmpeg inside is compiled to WebAssembly and the files it reads and writes are, by default, Go objects in memory. The guest never notices. This post is about the piece that does the lying: the vfs bridge.

Two filesystem interfaces, neither of them a filesystem

The requirement (R-AF-2 in the spec, one of the project’s founding rules) is that a media job can run with zero host filesystem access. Not “a temp dir we clean up afterwards”. None at all. The pipeline this library was built for generates a reel’s video in one step and posts it in the next, and there’s no reason the bytes moving between two steps of the same program should ever touch a disk.

So Go already has a beautiful answer to “a filesystem that isn’t”: afero, the de-facto standard filesystem abstraction, whose MemMapFs is a complete filesystem living in ordinary memory. And wazero, the wasm runtime, has its own answer from the opposite direction: a sys.FS interface describing what a guest expects a filesystem to look like, syscall by syscall.

The bridge is the adapter between those two worldviews:

sysCfg.WithSysFSMount(vfs.New(fs), "/")

vfs.New takes any afero.Fs and presents it to wazero as the guest’s root. When FFmpeg makes a WASI syscall (path_open, fd_read, fd_write, fd_seek), wazero routes it to the mounted sys.FS, and the bridge translates each one onto the corresponding afero operation (vfs.go at b9b7d5e). The comment on the constructor states the payoff plainly: hand it a MemMapFs and the whole pipeline stays in memory. Hand it a BasePathFs or a real directory and the same code runs against actual files, which is exactly how the integration tests double-check the abstraction isn’t cheating.

Test the scariest behaviour first

Spec 0003’s most opinionated decision wasn’t in the code, it was in the ordering: the first test written for the entire bridge was seek-on-write.

Here’s why that’s the scary one. When FFmpeg writes an mp4 with +faststart, it writes the file front to back, then seeks backwards into the finished output to relocate the moov atom (the index that lets a video start playing before it’s fully downloaded). Most toy filesystem shims survive sequential writes and die the moment something rewinds and patches the middle of a file. If afero’s in-memory files couldn’t take that punch, the whole project was a dead end, so that test was the go/no-go gate before anything else got built:

// Pwrite writes at an absolute offset without moving the file offset
// (fd_pwrite) — the path the mp4 muxer uses to patch the moov atom under
// +faststart.
func (f *file) Pwrite(buf []byte, off int64) (int, experimentalsys.Errno) {
	if len(buf) == 0 {
		return 0, 0
	}

	n, err := f.af.WriteAt(buf, off)

	return n, experimentalsys.UnwrapOSError(err)
}

(file.go at 661688d.) It passed, the gate opened, and the rest of the bridge is properly dull: EOF is reported the way WASI wants it (a zero-byte read with no error, not an error called EOF), errno values are mapped through wazero’s own error translation so the guest sees proper POSIX numbers, and anything the bridge doesn’t implement fails loudly with ENOSYS rather than pretending. Dull is the point. A filesystem is a contract and it’s always the clause you skimmed that bites, so the small print is where all the work went.

The paths that aren’t in your filesystem

The interesting wrinkle is the handful of paths a C program expects that no caller’s filesystem would ever contain. The bridge intercepts these in OpenFile before they reach afero (the dispatch):

/tmp routes to a separate in-memory scratch filesystem. FFmpeg is entitled to its temp files, but guest litter has no business appearing in the filesystem you handed in. Your MemMapFs stays exactly as clean as you gave it.
/dev/null is a discard sink, because portable C code will write to it and portable C code must be humoured.
/dev/urandom hands back real randomness from Go’s crypto/rand. That one arrived later and the hard way, after the Matroska muxer hung forever waiting for entropy that WASI never provides. The bridge was the natural home for the fix: not a patch to FFmpeg, just one more fixture the environment was expected to have.

There’s a theme in that list. WASI deliberately gives a guest almost nothing, and every one of these overlays is the bridge handing back a single, controlled piece of the POSIX world the guest was written for. The filesystem, the scratch space, the bit-bucket, the randomness. All of it fabricated, and all of it behaving exactly the way a real one would… which is the only thing the guest ever bothers to check.

What you get for the deceit

The practical upshot: afmpeg’s tests create no temp directories and need no cleanup, a crashed run leaves nothing on disk because nothing was ever on disk, and a server can process untrusted media without granting the decoder so much as a directory. It’s the same trick paying off twice over. Painless tests, and a sandbox you can hand somebody else’s video without flinching.

FFmpeg, for its part, never suspects a thing. It opens files, seeks around, patches its moov atoms, litters its /tmp… and every last byte of it is a Go map pretending, very carefully, to be a hard drive.

One input, many outputs

Thu, 20 Aug 2026 00:00:00 +0000

When I introduced afmpeg and ffmpeg-wasi, the post carried a deliberate caveat: “single input to single output and Probe work now; the full multi-pad filter_complex and multi-output muxing are the next thing on the bench.”

That was the true state of the engine on announcement day. So this is the follow-up: the thing on the bench is finished now, and how it works turned out to be the tidiest part of the whole project.

Why one output was never going to be enough

The whole reason ffmpeg-wasi links the libav* libraries directly instead of wrapping the ffmpeg CLI is control over exactly this sort of thing. A media pipeline in the real world rarely wants one file out. My reel pipeline wants the full-quality master and a lightweight preview; an audio workflow wants the loud mix alongside the quiet one. With the CLI you’d shell out twice and decode the input twice over… but with the graph in your own hands, one decode should be able to feed the lot.

FFmpeg’s filter-graph machinery has supported this forever: split (video) and asplit (audio) are filters whose entire job is duplicating one stream into several identical ones, so different processing chains can each take a copy. The engine just had to grow up enough to use them.

Two commits, two halves of the problem

It landed in two distinct steps, and the split between them tells you where the real work was.

The first rebuilt the processing core around the real filter-graph parser: N inputs feed one graph via avfilter_graph_parse2, and every labelled output pad gets its own encoder. That’s the “multi-pad” half, and validating it flushed out genuine bugs (an undeclared function that trapped the wasm outright, and a pts discontinuity that broke xfade). But every encoded pad still funnelled into a single output file. Half the promise.

The second is where the fan-out happens: one AVFormatContext, FFmpeg’s per-file muxing context, for each output, and a router that decides which file each graph pad belongs to:

static int find_output_for_pad(Ctx *c, const char *label) {
 for (int i = 0; i < c->n_out; i++) {
 const cJSON *m = NULL;
 cJSON_ArrayForEach(m, c->out[i].map) {
 if (cJSON_IsString(m) && label_matches(m->valuestring, label)) return i;
 }
 }
 if (c->n_out == 1 && (!c->out[0].map || cJSON_GetArraySize(c->out[0].map) == 0)) return 0;
 return -1;
}

Each output in the job declares a map, the list of pad labels it wants, and every pad is matched to its home. The fallback line keeps the old behaviour intact: a single output with no map takes everything, so every existing single-output job carries on untouched. From there, each encoder drains into its muxer rather than the muxer:

static int drain_encoder(Ctx *c, GOut *go, AVFrame *frame) {
 AVFormatContext *ofmt = c->out[go->out_idx].ofmt;

Headers are written per output, trailers are written per output, and split/asplit joined the enabled filter set in the same commit. Shipped as n8.1.2-5.

One decode, two files, one pass

The proof lives in afmpeg’s integration suite as TestIntegration_RunJob_MultiOutput. One WAV goes in. The graph is:

[0:a]asplit=2[a1][a2];[a1]volume=0.9[loud];[a2]volume=0.1[quiet]

asplit doubles the decoded audio, one copy gets turned up, one turned down, and two mp4 files come out of a single RunJob, each output’s map claiming its pad. The input was decoded exactly once. The test cracks both files open and checks they’re real mp4s, not empty files that merely exist.

Worth being precise about the shape of this: the fan-out is explicit. Every output names the pads it wants, and a pad nobody claims is an error rather than a guess. I’ve spent enough time debugging tools that helpfully infer things (the ffmpeg CLI’s own stream-mapping heuristics among them) to want the boring, spelled-out version in an engine that runs unattended.

The bench is clear

There’s a certain neatness in closing a gap your own announcement admitted to. Funnier still, the first half had already landed the morning the announcement went out, and the second followed two days later… cleared almost before the ink dried on the caveat. As a way of doing release notes, “under-promise, then ship anyway” beats the usual arrangement.

One decode in, as many outputs as the job asks for. The reel pipeline gets its master and its preview for the price of a single pass, and the engine’s job-spec grew the one word (map) it needed to say so.

A progress bar without a CLI to scrape

Fri, 25 Sep 2026 00:00:00 +0000

If you’ve ever wired a progress bar to FFmpeg you know the trick, and it’s a good one: run the CLI, watch stderr, and pull the numbers out of the frame=... fps=... time=... speed=... line it rewrites every half second. Near enough everyone does it, and each wrapper library I’ve looked inside does it too.

I wanted one for keryx, which renders its reels through afmpeg, so I went looking for the streaming stderr writer afmpeg hadn’t built yet. It hadn’t, and that wasn’t the problem.

There’s no line to scrape

afmpeg isn’t running the CLI. It drives a headless libav engine, compiled to WebAssembly by way of ffmpeg-wasi, and that engine writes to stderr on error paths and nowhere else. The encode loop, receive a packet, write a frame, receive a packet, emits nothing per frame at all. So with a perfect stderr stream in place there’d still be no signal on it to watch, and building the writer first would have got me a very good window onto an empty room.

So I stopped looking at stderr. There had to be some other buffer or mechanism to watch, and the shape I wanted was obvious even before I knew where the numbers would come from: a goroutine pushing progress back over a channel. A short spike first, to find out whether it was possible at all, and then a proper spec.

That instinct turned into the design. Once you’re no longer running the CLI, copying the CLI’s progress model is cargo-culting. The engine has the truth about where it’s up to, so the job is to surface that on purpose, as an API with a contract, and to keep the fallback honest when the engine can’t say.

Watching the bytes go past

The spike found the first source in an afternoon, and it needed no change to the engine at all.

afmpeg is the engine’s filesystem. The point of the thing, as I’ve written before, is that FFmpeg believes it has a disk and afmpeg is what’s underneath, calling the caller’s afero.Fs read by read and write by write. So the host can watch input being consumed and output being produced in real time by wrapping that filesystem, and the fraction is simply bytes read over input size. The spike gave 808 read events across a 45-second AAC encode, a smooth, monotonic climb from nought to a hundred percent, entirely through the public API.

That’s phase A in the spec, and it shipped first because it satisfied the need now.

It’s byte progress rather than time, and the spec is plain about where it’s poor: a demuxer that seeks (an mp4 with its index at the end) makes the raw count wander, so the reported fraction is clamped to the maximum seen and can’t go backwards; a generated input, a lavfi source with no file behind it, produces nothing to watch; and a very short job is over before there’s anything to say. For those the fraction is reported as -1, which means “can’t tell”, and the elapsed time and output bytes still come through so a consumer can show a spinner instead of a lie.

The engine gets a side channel

Phase B is where the engine joins in, and it comes in by the same route. The engine already talks to synthetic devices in that filesystem (/dev/null, /dev/urandom), so it gained one more: a write-only /dev/afmpeg-progress that the driver opens from its encode loop and streams newline-delimited JSON records into, one per flush interval, with the frame count and the output timestamp. The host is the filesystem, so it sees each write as it lands, parses the line, and feeds the same reporter that phase A was already using. Best-effort in both directions: if the device won’t open the engine carries on without it, and a consumer that doesn’t drain the channel misses samples and doesn’t stall the encode.

Two things in there I’d do the same way again. Speed is worked out on the host, output time over elapsed, because a WASI engine has no clock worth trusting. And the transport I rejected was the one that would’ve felt most natural to anyone who’s scraped FFmpeg before: an av_log callback formatting a stats line to stderr. It would have recreated the coupling the exercise set out to remove, just with the line coming from my code instead of theirs.

The caller sees none of the plumbing.

There’s one Progress value, one channel attached to the context with WithProgress, and the same call to Run as before. Phase B filled in fields the type had been carrying empty since phase A, and no signature changed.

Then it lied

Then I wired it into keryx, put a real reel render through it, and the fraction read 1.000 on the first sample and stayed there.

For the whole render.

That’s worse than the -1 the contract reserves for “can’t tell”. A caller can’t distinguish it from being finished, so the bar sat at done for the entire render, and a naive “did we get samples?” test passed. Two things had gone wrong at once, and the second one took me a while to see.

The first is that the byte ratio saturates when the inputs are tiny relative to the output. A reel is a handful of small PNG cards and a WAV bed, crossfaded into a thirty-second H.264 file, so the inputs are exhausted almost immediately and the denominator, which is only discovered as inputs open, tracks the numerator all the way up. I watched it read 367 of 367, then 734 of 734, then 32,778 of 32,778. The spec had even predicted it, in a sentence that said the effect was “safe, but lumpy” and that phase B’s engine fraction would remove it.

It didn’t.

That’s the second thing. Both sources fed through the same monotonic clamp, and a clamp is a shared maximum, so once the byte ratio touched 1.0 the ceiling was pinned there for good and the engine’s honest 0.033 had no way to pull it down. Upgrading the engine didn’t fix it.

I tried that first, reasonably enough, and spent a while convinced the engine was the gap… when the engine was supplying the right number and the host was throwing it away. My test suite missed the collision because the phase B test used a generated input with no bytes to count, so the two sources never met.

Spec 0034 is the fix, and it’s mostly a matter of deciding who wins. The engine’s number is authoritative when it exists. The fraction reports -1 during the startup window before the first engine record, instead of a byte ratio the engine is about to contradict, and -1 again for the tail where the inputs are all read and the job is still encoding. And the value now carries a Source field, bytes or engine or unknown, so a caller can decide how far to trust it.

On the native engine (there’s a post about that one tomorrow) the side channel goes quiet, because the device lives in the WASM backend, and progress falls back to the byte source. The native side channel has a spec of its own, blocked for now.

An honest -1

I started this wanting a progress bar for keryx and I got one, and what I’d carry to the next thing is the contract underneath it: a number that doesn’t go backwards, a -1 when the truth isn’t available, and a source label so you know which truth you’re looking at. A bar reading 100% for thirty seconds is a bug you’ll ship without noticing, because it looks like success, whereas a bar admitting it doesn’t know has at least told you something true, and the spinner it earns is the right thing to show.

The stderr line would’ve given me none of that, and it would’ve been working by lunchtime.

Sandboxed isn't safe

Thu, 03 Sep 2026 00:00:00 +0000

When I introduced afmpeg, one phrase did a lot of heavy lifting: safely process untrusted media. The FFmpeg inside is compiled to WebAssembly and run in a sandbox with no host filesystem, no network, no environment. Feed it a hostile file and the worst it can do is fail. That was the pitch.

An external security review read that pitch, then asked a better question than I had: fail taking how much of the host with it?

Two different questions wearing one word

“Sandboxed” answers one of them: what can it reach? And on that score afmpeg was solid. The guest sees an in-memory filesystem it was handed and nothing else. A malicious file that manages to run code inside the guest has, for all its scheming, broken into a padded room. No door, no window, nothing worth taking.

But there’s a second question the word “safe” smuggles in without telling you: how much can it take? Here afmpeg had no answer at all, and that was the review’s headline finding, one I’d commissioned and then made a point of verifying against the code rather than taking on trust. The runtime set no memory limit. None. A wasm32 guest can grow its linear memory towards 4 GB, and a crafted media file is only too happy to help it get there: declare a 65535×65535 video in the header and libavcodec, doing exactly the job you asked of it, tries to allocate the buffers those dimensions demand. Inside the sandbox. Paid for by the host.

So the guest can’t read your files, can’t phone home, can’t escape… and can still get your Go process OOM-killed by the kernel. I’d sandboxed the code execution and said nothing whatsoever about the resource bill. My padded room, it turned out, was drawing its air from the rest of the building, and I hadn’t thought to check.

The second finding was the same disease with a different clock: nothing bounded how long an invocation could run. So a pathological decode loop, handed a caller using a background context, would sit on the runtime’s lock forever. Not a crash. A wedge. (There was a third one too, minor: some missing type guards in the engine’s job-spec parsing, defence-in-depth against a caller the design already trusts. Fixed in the engine for completeness.)

Ceilings, on by default

The fix (0f20cbb) gives the sandbox the second half of its job. Memory first:

	rtCfg := wazero.NewRuntimeConfig().
		WithCoreFeatures(runtimeCoreFeatures).
		WithCloseOnContextDone(true)

	if pages := memoryLimitPages(cfg.memoryLimitBytes); pages > 0 {
		rtCfg = rtCfg.WithMemoryLimitPages(pages)
	}

The default ceiling is 512 MB, generous for real transcodes and a rounding error compared to “all of it”. A guest that tries to grow past the ceiling gets a failed allocation inside the sandbox, which FFmpeg handles the way it handles any allocation failure: the job errors, the host shrugs, life continues.

Time second (same commit):

	// Impose the default deadline before locking, but only when the caller brings
	// none — a caller's own deadline is honoured as-is (spec 0027 §4B).
	if _, ok := ctx.Deadline(); !ok && r.timeout > 0 {
		var cancel context.CancelFunc

		ctx, cancel = context.WithTimeout(ctx, r.timeout)
		defer cancel()
	}

An hour by default, and it steps aside for whatever deadline the caller already set. Both knobs can be turned, and setting either to zero opts out completely, but the decision that actually matters is the one in the spec’s own words: a sandbox whose protections are off-by-default is not a sandbox. The library user who never opens the hardening docs, which is most of us, most of the time, me being exhibit A, gets the ceilings anyway. Anything you have to remember to switch on is a thing you’ll forget to switch on.

Found on paper, not in production

For the record, nothing OOM’d in anger. This gap was found by a review I paid for, corroborated line-by-line against the runtime before a word of spec got written, and closed the same week. That’s the least dramatic possible version of this story and exactly the version I wanted: the alternative draft, the one where a user’s server dies processing a wedding video, writes itself and is worse in every respect.

Still, I’d rather own the embarrassing half out loud. I shipped “safely process untrusted media” while the runtime would happily hand a hostile file every byte it asked for, because I’d let the word “sandboxed” answer both questions when it only ever answered one. What can it reach? Fine, covered. How much can it take? Not a clue. A sandbox has to answer both before that little phrase is true, and mine was answering half of it with a completely straight face.

An encoder on borrowed time

Fri, 28 Aug 2026 00:00:00 +0000

There’s a date circled in ffmpeg-wasi’s documentation: 2027-11-29. On that day the last US essential patent in the AVC pool expires, H.264 becomes just another bit of maths in America, and a whole paragraph of careful legal hedging in my docs gets to retire. Until then the LGPL build of ffmpeg-wasi ships an encoder that exists on sufferance. And if you’re going to ship something on those terms, you say so. Out loud, prominently, with the date attached.

This is the story of the most-requested capability in any media toolkit (making mp4s people can actually play) and what it costs to offer it in a permissively-licensed build.

Decode was never the gap

The two-variant scheme gives ffmpeg-wasi an LGPL floor and a GPL full-fat build. From day one the LGPL floor could decode H.264; FFmpeg’s own decoder carries no GPL strings. The gap was encode. FFmpeg’s blessed H.264 encoder is x264, and x264 is GPL: enable it and the whole artifact graduates to the GPL variant. So the LGPL build, the one a permissively-licensed Go library wants to lean on, could read the world’s videos and write none of them.

Enter openh264, Cisco’s H.264 codec, BSD-2-Clause licensed. Two build flags (--enable-libopenh264 --enable-encoder=libopenh264, no --enable-gpl in sight) and the LGPL artifact encodes H.264. Shipped in n8.1.2-2, in both variants.

If that sounds too easy… it is. The BSD licence was never the hard bit.

Copyright is one umbrella; patents are another

A codec has two legal layers, and they don’t care about each other. The copyright on openh264’s source is Cisco’s to give away, and BSD gives it away. The patents on the H.264 techniques themselves belong to a pool of rights-holders (administered by Via LA), and no source licence in the world can waive them.

Cisco’s famous move was to pay the pool’s royalties on everyone’s behalf… for the binary modules Cisco itself builds and distributes. That grant does not travel with the source code. My docs put it in a red box rather than a footnote: ffmpeg-wasi compiles openh264 from source for wasm32-wasi (Cisco doesn’t publish a wasm binary), so our artifact is not under Cisco’s umbrella. Anyone who tells you “it’s fine, Cisco pays” for a self-compiled openh264 has read the first half of the licence page.

What makes it tenable is duller and more precise, and the licensing docs spell it out: the pool’s terms are royalty-free beneath an annual volume threshold (the first 100,000 units a year), a horizon this project’s expectations sit comfortably under. And the docs make a plain commitment on top: if Via LA or any AVC rights-holder asks us to stop distributing the encoder, we pull it. No fight, no fundraiser. The docs also say, twice, that none of this is legal advice, and neither is this post.

The horse had already bolted

You might ask why I’d take on any of this for the permissive build when the GPL build already existed. The answer is that the patent exposure was already there. Patents don’t read your source licence: an x264 binary practises exactly the same claims as an openh264 one, so the GPL variant had opened whatever door there was to open. Declining to add encode to the LGPL floor wouldn’t have closed it; it would only have kept the safer variant less useful. Same risk either way, and turning encode down would’ve bought me nothing but a less useful build. Call that caution if you like. I’d call it cost with no upside.

So both variants carry the risk, both carry the caveat, and the caveat comes with a date on it. Rather than embed a list of patent numbers that would only rot in place, the docs link Via LA’s maintained roster and state the one fact worth remembering: the final US essential patent lapses on 2027-11-29 (other jurisdictions track a similar horizon). None of this is meant to hold forever. There’s a clock running on it.

Making Cisco’s code believe in WASI

Getting openh264 v2.6.0 to build for wasm32-wasi took a small patch and a shim: a __wasi__ path through its threading library, a single-threaded pthread stand-in compiled into the archive, a hand-written pkg-config file (upstream’s insists on -lstdc++ -lpthread, neither of which exists on wasi)… and one genuinely fun find: a function declared with one signature in C and defined with another in C++, an arity slip native linkers have silently forgiven for years. Wasm’s strict function typing forgave nothing, and trapped on the spot. Which makes the sandbox an accidental linter, catching a bug that had sat there unbothered for years.

A feature with a use-by date

It nets out about as well as this kind of thing can. The LGPL build does the thing everyone actually needs, the risk is written down where you can’t miss it, the escape hatch is pre-committed, and the whole lot comes with a sell-by date, after which there’s nothing left to hedge and it’s back to being maths. Fifteen months on the clock. That red box in the docs is really just a changelog entry with the date left blank.

The afternoon the agent earned its keep

Thu, 09 Jul 2026 00:00:00 +0000

There’s a kind of bug that’s worse than a crash. A crash at least tells you where. This one just… stopped. I asked ffmpeg-wasi to remux a clip into a Matroska file, no re-encoding, just repackage the same streams into an .mkv, and it never came back. No error, no output. The test harness eventually gave up and shot it.

The same job to an .mp4? A dozen milliseconds. Change the file extension and a blink becomes a heat-death-of-the-universe operation.

Normally this is the moment a war story starts costing me evenings. FFmpeg compiled to wasm32-wasi, running under wazero, reading and writing through an in-memory filesystem… that’s a lot of layers, any of which could be the one that loops, and the traditional way to find out is to build test rigs for each layer by hand and lose a week to the plumbing. I know what that costs because I’ve paid it before.

This time I didn’t pay it. The hang surfaced while an agent was implementing stream-copy for me, so rather than pull the work back onto my own bench, I told it to chase the bug too. What follows is the deduction it ran. My contribution was two decisions at two forks, and I’ll point at them as we pass.

It got it wrong first

I want to start with the wrong turn, because it’s the most instructive part.

The agent’s first theory was confident and detailed: it traced the guest’s filesystem calls, saw the Matroska muxer seeking back and forth rewriting its header, and declared the finalisation seek-backs the culprit, an I/O pattern my filesystem bridge couldn’t satisfy. Plausible. Specific. Wrong.

And here’s the bit that sold me: it then designed the experiment that killed its own theory. Matroska has a streamable mode that writes without any seek-backs at all, so it flipped that on… and the job still hung. Theory dead, evidence in hand, no sulking. I’ve watched humans (fine, me) defend a pet theory for two days past its expiry date. The agent binned its own in minutes and moved on to the thing you should always do when guessing stops working: bisect.

Ruling out the world, one build at a time

This was my first fork. The agent asked whether to keep poking at it in place or to bring in native tooling, and I chose native. So it built a real FFmpeg, same n8.1.2 source, same minimal configure flags, and started eliminating suspects:

It tested	Result	Which ruled out
Native FFmpeg, 64-bit, real files → mkv	works	the muxer / our configure flags
wasm backed by a real OS directory	hangs	the in-memory filesystem
Native FFmpeg, 32-bit (`-m32`) → mkv	works	a 32-bit integer overflow
wasm under wazero’s interpreter	hangs	a code-generation bug

Sit with that table for a second. Two of those rows are whole native builds of FFmpeg, one of them 32-bit with the toolchain to match, spun up as casually as you’d write a unit test. That’s the part that would have eaten my week. Each row walks an interesting suspect free, until the only thing left holding the bag is the plainest fact in the whole setup: this is running under WASI. Not the muxer. Not the maths. Not my filesystem. Not the runtime. The environment.

The environment was missing a number

WASI is a smaller world than POSIX. No threads, no network, and, it turns out, no /dev/urandom. Most code never notices. Right up until it does.

With the bisection result in hand, the agent went and read the Matroska muxer’s source. Every Matroska track carries a unique random identifier, so at startup the muxer seeds a pseudo-random number generator via libavutil’s av_get_random_seed(). In a build configured as tightly as ours, that function has exactly one source of real entropy compiled in: reading /dev/urandom. The Windows crypto API, arc4random, OpenSSL, libgcrypt… all configured out. Just the one device.

Which WASI doesn’t have. So the read fails, and the function falls back to its last resort, get_generic_seed(): a genuinely clever routine that manufactures entropy out of thin air by reading clock() in a tight loop and harvesting the jitter in how it advances. It’s a lovely trick. It has one requirement. The clock has to advance.

Under WASI, clock() doesn’t advance. The seeding call never returns. The muxer hangs before it has written a single byte, and the .mp4 muxer never asks for randomness at all, which is the entire reason it was fine all along.

The agent didn’t leave that as a theory either. It wired av_get_random_seed() into a trivial diagnostic op, one that does nothing else… and that hung too. Root cause, demonstrated in isolation.

A device, not a patch

The fix isn’t a patch to FFmpeg. It’s a device. My filesystem bridge already fabricates a couple of the POSIX fixtures a guest expects (/tmp routed to scratch space, /dev/null as a discard sink), so /dev/urandom is the same trick with real randomness behind it, backed by Go’s crypto/rand:

// Read fills buf with random bytes; a short read never happens (crypto/rand
// fills the whole slice or errors), matching /dev/urandom's contract.
func (*randFile) Read(buf []byte) (int, experimentalsys.Errno) {
	if _, err := rand.Read(buf); err != nil {
		return 0, experimentalsys.EIO
	}

	return len(buf), 0
}

Now the muxer’s very first entropy read succeeds, the seed returns immediately, and the clever-but-doomed clock loop is never reached. The .mkv remux that had been hanging forever finishes in ten milliseconds. And the fix isn’t Matroska-specific: any format that wants a random seed now gets one for free.

What the afternoon actually cost

Here’s the arithmetic that made me want to write this up.

From the first observed hang to a verified fix was a little over an hour. That hour included the wrong theory and the experiment that disproved it, two complete native builds of FFmpeg (one of them 32-bit), runs across two wazero execution modes, a read through libavutil’s entropy code, the diagnostic op that proved the root cause in isolation, and the fix itself. The agent even went looking for prior art and couldn’t find anyone who’d written this failure up… a genuinely undocumented corner of WASI, run to ground between lunch and tea.

By hand? Most of a week, and the bulk of it spent on build plumbing rather than thinking: getting a matching native FFmpeg to configure, wrangling a 32-bit toolchain, wiring the trace points. None of that work is hard. It’s just time, and it’s exactly the kind of time a solo developer doesn’t have lying around, which is how deep bugs like this end up parked on a “someday” list with the .mkv button greyed out.

My whole part was two forks (dig in place, or go native? and later: document it before touching anything else) and reading the result. The judgement stayed mine. The legwork didn’t.

I’ve compared these agents to Golden Retrievers before, and I stand by it: eager, tireless, loyal to their training, occasionally proud of a shredded cushion. A dog that plays fetch all day is good company. But this one went down the burrow after something I couldn’t see, and came back with the exact rabbit.

Good boy.