This started because keryx needed to make videos, and I didn’t want to give it a disk.

That is honestly the whole of it. keryx builds reels, reels need FFmpeg, and I wanted an FFmpeg I could point at a virtual filesystem instead of the real one, something that reads and writes through an afero filesystem I handed it and never learns that a hard drive exists. A small ask, I thought, late one night at my desk… with a rather large problem hiding behind it.

FFmpeg is a CLI, and that’s the whole trouble

FFmpeg is magnificent and it is a command-line tool. So when an engineer needs it, they shell out to it, and two things follow from that (neither of them anyone’s fault).

The first is that you inherit FFmpeg’s arguments, which are a language of their own and not a friendly one. Most of us who have used it in anger have a shell script somewhere with a line in it we no longer understand and daren’t touch (I have several).

The second is the one I actually cared about. Shelling out means handing a process the run of the machine. It opens paths, seeks around, writes where it likes. If you’re processing something a stranger uploaded, that’s a decoder with your permissions and your filesystem, and “it’s fine, it’s just FFmpeg” is doing a lot of work in that sentence.

I’ve prided myself on being a security-first engineer for a long time, and that arrangement has always sat badly with me. What I also knew, and it’s the only reason I thought this was possible at all, is that you don’t have to talk to FFmpeg through a CLI.

Fifteen-odd years ago, as a PHP engineer, I used an extension that drove FFmpeg directly, with no shell and no arguments, just calls. So the shape existed already. Just not in Go.

Deliberately ignorant of the hardware

The answer was to compile FFmpeg to WebAssembly and run it as a guest, reading and writing through the afero filesystem I’d handed it. That became ffmpeg-wasi and afmpeg, and the filesystem trickery has its own post.

The important property is one it’s easy to mistake for a limitation: afmpeg has no idea what hardware it’s on. It doesn’t ask and it doesn’t care. Hand the same binary to a laptop, a CI runner or a container with nothing in it, and you get the same answer, because it never assumed a thing about the machine.

Not needing hardware is what makes it portable. That’s the design, and it stays the design. Where it got awkward was the day I wanted the hardware myself.

I was hedging even then

When the hardware-acceleration question first came up, I wrote the premise into a research prompt so it could be checked properly: Wasm implementations, in their current form, can’t reach hardware. And then I tacked “please correct me if I’m mistaken” onto the end of it.

Just as well. The tidy version of that claim is wrong. WebAssembly in a browser reaches the GPU perfectly well. That’s what WebGPU and WebGL are: host APIs the browser hands the module. The sandbox doesn’t grant that access, the host does, and that’s what a sandbox is for.

Outside a browser, same rule. The runtime has to offer it, no WASI standard obliges anybody to, and a runtime that wanted to would be binding to native graphics libraries to get there. afmpeg runs on wazero, which is zero-dependency and pure Go, and there’s a line in the justfile building the whole library with CGO_ENABLED=0 on every run so that promise can’t rot.

So the position was consistent. It had also, without me ever deciding it should, become a corner. There’s a difference between not requiring hardware and not permitting it, and afmpeg had the first only by having the second. So the plan was a small escape hatch: narrow, heavily guarded, and apologised for in the documentation.

Then I asked a much better question

The thing that changed it was noticing what I’d already built. afmpeg doesn’t consume anyone else’s FFmpeg. ffmpeg-wasi compiles its own, with its own driver interface, because making FFmpeg work under WASI meant building that interface anyway. It’s mine, and I ship it.

So could I layer an alternative interface onto the same thing, and distribute that variant too? The answer took an embarrassingly long time to land (weeks, if I’m honest, for something that now looks obvious). The interface built to make FFmpeg work in a sandbox is not sandbox-shaped at all. It’s just a clean way to drive FFmpeg without a command line, and if you point it at a native build instead of a Wasm one it works the same, minus the guest.

That is how spec 0028 stopped being about an escape hatch and started being about a second engine. The native driver runs as a separate process and is served the caller’s afero.Fs over a Unix socket, so the filesystem promise holds on both sides:

func (b *Backend) Invoke(ctx context.Context, fs afero.Fs, args ...string) (afmpeg.Result, error)

You still hand it a filesystem. It still can’t go looking for one. Swapping engines is one option, WithBackend, and the job API doesn’t change at all: same calls, byte-compatible results.

And it is significantly faster. I don’t mean marginally, or within noise. I mean the reel job stopped being a thing I started and then went to put the kettle on for, and became a thing that had finished before I’d stood up!

I’m deliberately not giving you a multiplier here, and there was one in an earlier draft of this. I’ve taken it out. The figures are being re-measured properly, on a quiet machine and against a couple of defects that were found in the meantime, and I suspect a single headline number is a poor way to describe what’s actually going on anyway. There’ll be a post with the full breakdown once the measurements are worth trusting.

For now: significantly faster, on the jobs keryx actually runs, which is where this all started. And because the native driver is a native binary, with real asm and real threads, the hardware is reachable from that side in a way it never was from the guest. I haven’t wired it up or put it through its paces yet, but it’s there, which is a good deal more than the corner was offering.

I’d intended a hatch.

What I’d actually built was a set of french doors… and I could throw them wide open!

The sentence I made myself write down

Running FFmpeg as a native subprocess is an objective security regression compared to WASM. I can’t spin that as a trade-off, or as “different security properties”. It’s worse.

The process boundary is real and the socket is a real constraint, but a native binary executing on your host with your permissions is not the same animal as a Wasm module that physically cannot make a syscall you didn’t hand it. That went into the documentation in those words, so anybody switching engines does it knowing what they’ve put down. A door you don’t have to answer for is a hole with a nicer name.

Still not the bottom rung

Having said all that, and meaning every word of it, there’s a comparison worth making. The alternative most people reach for isn’t WASM. It’s the shell.

Set the native driver against exec.Command("ffmpeg", ...), which is what the overwhelming majority of media pipelines actually do, and it comes out ahead on the thing that matters. Shelling out hands a process the run of the machine and a set of paths you typed. The native driver gets a socket, and on the other end of that socket is the afero filesystem afmpeg gave it: a filesystem you built, holding what you decided to put in it, with no directory to climb out of.

So the ladder runs like this. Shelling out at the bottom, with your whole disk in scope. The native driver in the middle, sandboxed at the filesystem even though the binary is native. WASM at the top, where the guest can’t reach anything at all without being handed it.

The middle rung is a step down from the top, and a long way up from where most of this work gets done. The default stays ignorant (runs anywhere, assumes nothing), and the hardware is a thing you open on purpose, and even then you don’t get the keys to the house.

Anyone can take it

ffmpeg-wasi is a standalone project. It isn’t a folder inside afmpeg and it doesn’t need afmpeg to be useful. It’s a custom FFmpeg build with a proper driver interface, distributed on its own, so any language with C or FFI bindings could drive the thing directly.

I’d be delighted if they used afmpeg as well, obviously. But they don’t have to, and that wasn’t the plan either. The plan was to stop keryx from having a disk.

Somewhere along the way it turned into an FFmpeg you can embed without shelling out, in a sandbox by default, at native speed when you ask, in whatever language you fancy. I still can’t tell you what half the FFmpeg arguments do!