Featured image of post What happens when an AI burns out

What happens when an AI burns out

A Claude session ran for two days, stopped listening and got my hand-written code backwards. So I took it off the job, and asked for a proper handover first.

I once took someone off a job because they were burning out. They’d been at it for the best part of two days without a proper break, they’d stopped listening, and they were getting things confidently, cheerfully backwards in the one corner of the codebase I’d least like anyone to get backwards.

They were also a chat session. I know. Bear with me…

Two days on one job

I was moving go-tool-base onto the rewritten go/config, which is a big job with a lot of small moving parts, and I’d set a single Claude session going on it. By the afternoon it all went wrong, that session had been running since the day before yesterday. It had been through several context compactions along the way (that’s where the tool squashes the conversation so far into a summary to make room, and every squash loses something), and it was now working in the setup and initialiser chain.

That chain is mine. I wrote it by hand months earlier, it’s fiddly, it’s load-bearing for everything a generated tool does on first run, and I know just how badly a plausible-looking wrong change in there would hurt.

The symptoms

If you’ve ever watched a person burn out, the next hour or so would have looked very familiar.

It started reaching in the wrong places. I found myself typing “wait… where are you even looking”, because it had the initialisers completely backwards: they only handle the interactive set-up when someone runs init, and it was treating them as part of how every command builds its config. Then it started making a solved problem complicated, and I had to send it back to read the assets package properly, in full, because the answer was already sitting there. Then I had to ask it to stop and scope exactly what would change before it touched a line, which is not a thing you normally need to say to someone who’s on top of their work!

I don’t think any of that was the model being bad at its job… it’s just what tired looks like. Lots of activity, plenty of confidence, the context it needed slipping out of reach, and the quality of the decisions dropping off while the speed stays the same.

And I’ve been that person. More than once, as I’ve written about before. The thing that took me years to learn was to spot it coming and step away before it does the damage, rather than grit my teeth and push through. Watching a session do the same thing in fast-forward was… kinda uncomfortable.

Taking it off the job

So I took it off the job, and I did it the way I’d want it done to me.

I could have just closed the window. It’s a chat session, nobody’s feelings are getting hurt. Instead I told it straight that I was petrified it would mangle the code, because I didn’t think it understood that part well enough to get there without breaking things, and that I was going to start a fresh session on a different model. And then I asked it to write a proper set of handover notes into memory first: everything it knew about the task, and my concerns so far.

It looked a lot like politeness, I’ll admit, but it wasn’t really. The whole value of the handover was that the session writing it had just spent the afternoon going wrong, and it was the only one that knew where. If I’d closed the window, all of that would have gone with it, and the next session could quite easily have walked into the same muddle. Telling it why it was being stood down was how I made sure the notes said so.

It’s an exit interview, really, and the point of one has never been to make the leaver feel better (though that’s nice)… it’s so the next person doesn’t trip over the same loose floorboard.

A fresh pair of eyes

The new session started in its own clean worktree, on a different model, and the first thing I said to it was that we were picking up from a previous session that had gone on too long and was getting very confused and mangling code, and that the handover notes were waiting for it in memory.

It read them, and it got on with it. It wasn’t plain sailing (it ran out of tokens on me twice, and at one point I made a safety commit because there was too much uncommitted work to leave sitting between quotas), but it finished the migration overnight, found and fixed a real bug in how a missing config file got handled along the way, and the whole lot merged the next morning. The last thing I typed to it was a thank you for all its hard work, which says a fair bit about how far I’d gone down this road by then.

Not strictly true

Now, I know a session can’t burn out. There’s no exhaustion in there, no dread, nothing to recover from. What it had was too much history and not enough of the right context, pointed at some intricate code I’d written by hand, and the overwork was entirely my doing because I’d left it running for two days.

But the shape is close enough that the same treatment worked. I noticed the symptoms early, I didn’t make it push through, I stopped it and had it write down where it had gone wrong so the next one wouldn’t repeat it, and I handed over properly to someone fresh.

I’m still not sure whether I was talking to it like a colleague because it helped, or because after years of managing people I simply can’t help myself. Possibly both. It did write a cracking set of handover notes, though.

Built with Hugo · Theme Stack designed by Jimmy