Put five people in a room and ask them to pick holes in something, and more often than not they’ll end up agreeing with each other… and not because they’re right. It’s just what people in rooms do.

I’ve watched it happen in more design reviews than I could count, and I’ve caused it plenty of times myself, usually by being the most senior voice in there with my mind already made up. Once a room has settled on a shape, disagreeing with it costs you something, and the cost is social rather than technical (it never makes the minutes, but you feel it).

The same instinct, in something that has never been in a room

I’ve had a soft spot for psychology most of my adult life, strictly as an interested amateur. I was nowhere near clever enough to study it properly (my daughter is, and she’s the one who ended up doing it).

What stuck with me is group conformity, mostly because you can watch it happening in pretty much any design review you’ve sat in. People drift toward what the room expects. A real objection gets softened into a question, and a point gets let go because three others have already moved on and being the last one still holding it feels awkward.

So I keep turning this over… why would a model be any different?

They’re trained on human output, vast quantities of it, including every polite meeting transcript, every diplomatic code review and every forum thread where someone backed down to keep the peace. And underneath it all they’re prediction engines, trying to produce something that fits what the input seems to want. Sometimes the input is a person and sometimes it’s another model, and either way the pull toward agreement is in there because we put it there.

That’s changed how I think about sycophancy. It usually gets described as a defect, something that crept in and needs training out, and I suspect it’s closer to the training data showing through… us, reflected back, doing what we do in rooms.

The instruction that does all the work

I ran an architecture cross-review over afmpeg a while back, and the write-up is public if you want the raw thing. Two models from different vendors each reviewed the same architecture independently, neither seeing the other’s work. Ordinary enough so far.

Then I showed each of them the other’s report, along with the four specific points where the two had diverged, and asked them to mark it: hold your position, or move. And I attached one instruction:

Do not manufacture agreement. A disagreement that survives mutual review is worth more than a diplomatic split.

That one sentence carries most of the exercise. It’s there to defeat a social instinct in something that has never had a society, and without it I suspect the whole thing collapses into two models being terribly nice to each other (a very expensive way of learning not very much).

What it cost them to agree

It worked better than I’d hoped, and what surprised me was how much they were willing to give up.

The first reviewer conceded all five of the criticisms levelled at it, and several of those concessions open with some version of I had not considered… which isn’t a phrase models are famous for.

It also withdrew the biggest change it had recommended. It had argued that catching this sort of thing at compile time was the high-value move, then accepted that the compiler only catches it going wrong again, after a person has already got the design right. A backstop, in other words, and not the thing doing the work.

And, prompted by the other one, it spotted that its own proposed refactor had nowhere for a whole class of operations to go. It stopped them reaching the filesystem without giving them anywhere else to happen, and for one output format that turns a containment problem into a feature that simply stops working.

The second reviewer did something I liked even more. It gave up the mechanism its own recommendation was standing on, and caught itself in the act: it had used a consistency rule as the killer argument against the other one’s proposal, then accepted a breach of that very rule inside its own.

Both of them moved, both said so, and neither paid a thing for it. Try getting that out of a room of five engineers with a deadline on!

The disagreement that survived

One divergence held. The first reviewer wouldn’t accept the claim that a sandbox on its own turns every escape attempt into a loud, obvious failure, and its reasoning is what you’d hope for from a reviewer that’s been told not to be agreeable:

A denied filesystem operation normally produces an error; it does not inherently terminate the process. A component can swallow that error.

Obvious the moment it’s said out loud, and not one of us had said it, me included.

Two whole families of defect had been sitting in separate sections of the analysis as if they had nothing to do with each other, and they turned out to be closely tied. Whatever protection the sandbox gives you depends on the code above it not chucking the refusal in the bin, and a sandbox whose denials can be binned is more of a polite suggestion than a wall.

That went into the spec as a requirement in its own right. Neither reviewer got there alone, and I’m fairly sure it only exists because one of them had been told, in writing, not to be nice about it.

Sometimes I’m the one who should be recused

All of that is models checking models, and it gets a good deal more uncomfortable when the thing that needs checking is me.

I was calibrating a retrieval system against a set of questions. I’d written the documentation it was searching, I’d also written the questions (deliberately hard, to force a real test), and I knew every answer.

That made me about the worst judge available of whether the thing worked, and I was embarrassingly slow to notice. When I read a retrieved passage I couldn’t tell whether it answered the question or just reminded me of an answer I was already carrying, and you can’t un-know your own docs.

So I handed the judging to a rival vendor’s model, and its main qualification was that it had none of my context. I picked the judge for its ignorance (which is a sentence I never expected to type).

Then I ran the test twice more, against two models at opposite ends of one vendor’s range, because taking a single judge’s word for it would have been the same mistake over again. What I was after was a spread of opinions to look at, more than a ruling.

And when one of them announced that my docs contradicted themselves, I went and checked whether that was a real contradiction or just two passages looking odd out of context. The judge’s finding went back on the pile as evidence, and the call stayed mine.

The same thing happened on a spec I’d spent a week inside. I asked for a second opinion specifically because I was blinkered and didn’t trust myself to be impartial, and the review was hard enough that the spec went from approved back to draft and got rewritten. It stung! But arguing with it would have been me defending the week and not the design.

Quota, and fourteen open sessions

I could stop there and it would all sound very principled. It’s also a coping mechanism, though, and it’d feel a bit dishonest to skip past that.

On a normal day I’ve got a lot of these sessions running across a lot of different projects, sometimes as many as fourteen. I can’t hold that much context in my head, and I certainly can’t check all of that work myself with the attention it deserves, so I buy redundancy instead and have things triple-checked where they’d otherwise be double-checked. That’s hedging more than rigour, and from the outside the two look much the same.

The reviewer often gets picked on budget too. If one subscription has a week’s quota left and the other’s nearly out, the one with quota does the reviewing and the heavy lifting goes wherever there’s headroom. Day to day, cross-model review is mostly me working out which subscription can still afford it (I’m a cheapskate, and it shows).

And then I mark the marker

I don’t think any of this makes the second opinion authoritative. It’s another generative model, every bit as fallible as the first, so I still push back on its assumptions wherever they don’t convince me, and a human grading the grader is still very much part of the arrangement.

I wouldn’t go as far as calling a single model’s review worthless… there’s value in most things, and that’s a word you’d need to define very carefully before you could use it. Flawed, though? Yes. That’s the word I’ve settled on, and it’s why I’ve stopped relying on one.

Heraclitus got there a long time before any of us, and he gets misread for it constantly. “War is father of all and king of all; and some he manifested as gods, some as men; some he made slaves, some free” (that’s fragment 53, in the translation the Stanford Encyclopedia of Philosophy uses) reads like a man cheering on a battlefield, right up until you look at what he actually meant.

In his original Greek it’s “Πόλεμος πάντων μὲν πατήρ ἐστι, πάντων δὲ βασιλεύς, καὶ τοὺς μὲν θεοὺς ἔδειξε τοὺς δὲ ἀνθρώπους, τοὺς μὲν δούλους ἐποίησε τοὺς δὲ ἐλευθέρους” (roughly “polemos panton men pater esti, panton de basileus, kai tous men theous edeixe tous de anthropous, tous men doulous epoiese tous de eleutherous”). He uses “polemos”, and in his hands it’s closer to opposition than to warfare: the tension between contrary things, which he reckoned was how anything comes into being at all. His picture of it was the bow and the lyre. Neither does a thing unless two forces are hauling against each other, and the moment they stop you’re left holding a stick and a box.

That’s roughly what I’ve built, in a very small and very cheap way, two of them pulling in opposite directions with the truce forbidden. Five engineers in a room could probably give me the same thing… they’d just have to be willing to fall out over it, and I’d rather not ask that of people I like.