Featured image of post A terrible lead to an AI junior

A terrible lead to an AI junior

I'd built this exact thing by hand, years ago, pixel perfect. So why was I sitting here shouting at a screen while a machine got it wrong three times?

I was shouting at my screen. Out loud, in an empty room, at a text editor (I really was).

Three goes in, and the thing still couldn’t put a caption where I’d dragged it.

I have built this exact thing before

The feature is easy to describe: grab a caption in keryx’s studio, drag it where you want it on the panel, and have the rendered MP4 agree with you. A WYSIWYG editor sat over a video renderer. And I have done it before, properly, by hand, a very long time ago.

I was at an agency that had been brought in to build a greetings card website. Our client was an upstart with ambitions, and the behemoths they were trying to topple were Moonpig and Funky Pigeon. You picked a card, you dragged your text and your photo about on a preview, and what came out the other end had to be the thing you’d actually see when it dropped through someone’s letterbox.

They came in the door and asked for it in Flash, like the others had. And I, a young whippersnapper absolutely brimming with confidence, said no. Flash was legacy, it was turning into a security problem, and everything they were describing was perfectly possible in JavaScript, because all of it, every last bit of it, is just maths!

This was long before AI was any use to anybody (and back when jQuery was still a reasonable thing to reach for), so it was hand-crafted, every line. And it was pixel perfect: every element positioned on that preview landed just where it should on the final render. Took me a couple of days (and I’ve clearly dined out on it since).

So why was this taking three goes?

That’s why I was shouting. I knew how this worked, I’d known for close to twenty years: absolute positioning and a scaling ratio. It isn’t rocket science, and I’d done it under deadline with worse tooling and a lot less experience. An AI ought to knock that out in minutes.

Instead, three rounds. Each time I’d render a real reel, watch it back, and write down what was still off, and each time the thing that got fixed stayed fixed while something new stepped up to take its place.

First the text boxes jumped and resized while I dragged them. Fixed. Then the dragging went smooth, and the font size was perfect in the preview and had no fidelity whatsoever to the render. Fixed that too… or thought I had. Then the alignment came out right vertically and still wrong horizontally.

That last one is the tell. Vertical correct, horizontal wrong, out of the same code doing the same arithmetic on a different axis. That’s not a maths error. Maths errors don’t pick a favourite axis.

It was measuring. It should have been reading.

The ratio has two inputs, and it was measuring both of them! The preview size came from asking the browser how big the element had ended up, and the render size came from probing the encoded video file. Two measurements, both perfectly sensible on their own, both taken from things sitting downstream of the actual answer.

And that video wasn’t even the render, it was a scaled-down proxy made for the preview, so probing it doesn’t give you the target dimensions at all, it gives you the proxy’s plus whatever the encoder decided about even numbers. Ask a browser for an element’s size, meanwhile, and you get sub-pixel layout, device pixel ratio, and a rounding decision nobody wrote down. Second-hand, both of them, a copy of a copy, with very careful arithmetic being done on the result.

Meanwhile the real number, the actual target handed to the encoder, was sat in the codebase as a constant. Seven lines into internal/reel/timing.go:

// Geometry / timing defaults (parity with gen-reel.py).
const (
	Width  = 1080
	Height = 1920
	FPS    = 30

1080 by 1920. Defined by us, passed to the encoder by us, and never once consulted by the code trying to work out how big the render was going to be. It also explains the axis thing, since two dimensions pick up different rounding in different places, so of course one lands while the other misses.

We found it, we fixed it, and it’s been perfect since. And the principle underneath is worth having: when a value is defined by your own code, don’t go re-deriving it by measuring something downstream… read it.

The question I should have asked three rounds earlier

If I’d already solved this, by hand, more than once, why was I letting a machine solve it at all?

Laziness, mostly, and I’ll own that one. It looked trivial to me because I’d done it several times before, so it registered as boilerplate, and boilerplate gets handed off without a second thought. That’s the mistake, and it isn’t really a technical one: I gave it a list of requirements and let it get on with it.

No technical guidance, no “the target dimensions are a constant, they’re in timing.go, use those and don’t measure anything”, nothing about the solution I already knew worked, because to me it wasn’t a solution, it was just how you do it.

I was a terrible lead to a junior engineer!

Because that’s what happened. I briefed a capable junior on outcomes, gave it nothing on approach, and then got annoyed when it took the reasonable-looking path instead of the correct one. It didn’t have my twenty-eight years to draw on. It had my requirements.

The maddening part is that the thing I skipped is the thing I’d have given a human without a moment’s thought. You don’t hand a junior a ticket that says “make the preview match the render” and walk off, not if you’ve solved it before and know where the sharp edge is. You tell them about the sharp edge. That’s most of what leading is.

I did it in a couple of days once, with jQuery, on a deadline, while arguing a client out of Flash, all of which taught me absolutely nothing about how to explain it to someone else. Maybe if Musk gets his way with Neuralink I’ll be able to just think the context at it. Until then I need to be a better lead engineer for my junior.

Built with Hugo · Theme Stack designed by Jimmy