If you’ve ever wired a progress bar to FFmpeg you know the trick, and it’s a good one: run the CLI, watch stderr, and pull the numbers out of the frame=... fps=... time=... speed=... line it rewrites every half second. Near enough everyone does it, and each wrapper library I’ve looked inside does it too.

I wanted one for keryx, which renders its reels through afmpeg, so I went looking for the streaming stderr writer afmpeg hadn’t built yet. It hadn’t, and that wasn’t the problem.

There’s no line to scrape

afmpeg isn’t running the CLI. It drives a headless libav engine, compiled to WebAssembly by way of ffmpeg-wasi, and that engine writes to stderr on error paths and nowhere else. The encode loop, receive a packet, write a frame, receive a packet, emits nothing per frame at all. So with a perfect stderr stream in place there’d still be no signal on it to watch, and building the writer first would have got me a very good window onto an empty room.

So I stopped looking at stderr. There had to be some other buffer or mechanism to watch, and the shape I wanted was obvious even before I knew where the numbers would come from: a goroutine pushing progress back over a channel. A short spike first, to find out whether it was possible at all, and then a proper spec.

That instinct turned into the design. Once you’re no longer running the CLI, copying the CLI’s progress model is cargo-culting. The engine has the truth about where it’s up to, so the job is to surface that on purpose, as an API with a contract, and to keep the fallback honest when the engine can’t say.

Watching the bytes go past

The spike found the first source in an afternoon, and it needed no change to the engine at all.

afmpeg is the engine’s filesystem. The point of the thing, as I’ve written before, is that FFmpeg believes it has a disk and afmpeg is what’s underneath, calling the caller’s afero.Fs read by read and write by write. So the host can watch input being consumed and output being produced in real time by wrapping that filesystem, and the fraction is simply bytes read over input size. The spike gave 808 read events across a 45-second AAC encode, a smooth, monotonic climb from nought to a hundred percent, entirely through the public API.

That’s phase A in the spec, and it shipped first because it satisfied the need now.

It’s byte progress rather than time, and the spec is plain about where it’s poor: a demuxer that seeks (an mp4 with its index at the end) makes the raw count wander, so the reported fraction is clamped to the maximum seen and can’t go backwards; a generated input, a lavfi source with no file behind it, produces nothing to watch; and a very short job is over before there’s anything to say. For those the fraction is reported as -1, which means “can’t tell”, and the elapsed time and output bytes still come through so a consumer can show a spinner instead of a lie.

The engine gets a side channel

Phase B is where the engine joins in, and it comes in by the same route. The engine already talks to synthetic devices in that filesystem (/dev/null, /dev/urandom), so it gained one more: a write-only /dev/afmpeg-progress that the driver opens from its encode loop and streams newline-delimited JSON records into, one per flush interval, with the frame count and the output timestamp. The host is the filesystem, so it sees each write as it lands, parses the line, and feeds the same reporter that phase A was already using. Best-effort in both directions: if the device won’t open the engine carries on without it, and a consumer that doesn’t drain the channel misses samples and doesn’t stall the encode.

Two things in there I’d do the same way again. Speed is worked out on the host, output time over elapsed, because a WASI engine has no clock worth trusting. And the transport I rejected was the one that would’ve felt most natural to anyone who’s scraped FFmpeg before: an av_log callback formatting a stats line to stderr. It would have recreated the coupling the exercise set out to remove, just with the line coming from my code instead of theirs.

The caller sees none of the plumbing.

There’s one Progress value, one channel attached to the context with WithProgress, and the same call to Run as before. Phase B filled in fields the type had been carrying empty since phase A, and no signature changed.

Then it lied

Then I wired it into keryx, put a real reel render through it, and the fraction read 1.000 on the first sample and stayed there.

For the whole render.

That’s worse than the -1 the contract reserves for “can’t tell”. A caller can’t distinguish it from being finished, so the bar sat at done for the entire render, and a naive “did we get samples?” test passed. Two things had gone wrong at once, and the second one took me a while to see.

The first is that the byte ratio saturates when the inputs are tiny relative to the output. A reel is a handful of small PNG cards and a WAV bed, crossfaded into a thirty-second H.264 file, so the inputs are exhausted almost immediately and the denominator, which is only discovered as inputs open, tracks the numerator all the way up. I watched it read 367 of 367, then 734 of 734, then 32,778 of 32,778. The spec had even predicted it, in a sentence that said the effect was “safe, but lumpy” and that phase B’s engine fraction would remove it.

It didn’t.

That’s the second thing. Both sources fed through the same monotonic clamp, and a clamp is a shared maximum, so once the byte ratio touched 1.0 the ceiling was pinned there for good and the engine’s honest 0.033 had no way to pull it down. Upgrading the engine didn’t fix it.

I tried that first, reasonably enough, and spent a while convinced the engine was the gap… when the engine was supplying the right number and the host was throwing it away. My test suite missed the collision because the phase B test used a generated input with no bytes to count, so the two sources never met.

Spec 0034 is the fix, and it’s mostly a matter of deciding who wins. The engine’s number is authoritative when it exists. The fraction reports -1 during the startup window before the first engine record, instead of a byte ratio the engine is about to contradict, and -1 again for the tail where the inputs are all read and the job is still encoding. And the value now carries a Source field, bytes or engine or unknown, so a caller can decide how far to trust it.

On the native engine (there’s a post about that one tomorrow) the side channel goes quiet, because the device lives in the WASM backend, and progress falls back to the byte source. The native side channel has a spec of its own, blocked for now.

An honest -1

I started this wanting a progress bar for keryx and I got one, and what I’d carry to the next thing is the contract underneath it: a number that doesn’t go backwards, a -1 when the truth isn’t available, and a source label so you know which truth you’re looking at. A bar reading 100% for thirty seconds is a bug you’ll ship without noticing, because it looks like success, whereas a bar admitting it doesn’t know has at least told you something true, and the spinner it earns is the right thing to show.

The stderr line would’ve given me none of that, and it would’ve been working by lunchtime.