The Bug Was in the Instruction, Not the Code
In one release, the reviewers caught the coordinator's own instructions being wrong three times. Not the builders' code. The instructions.
I use AI agents to fix my assistant app. In Green Tests, Broken App, I ended with a rule: one coordinator plans and checks the work, builders from three vendors write the fixes, and every piece is checked by the two vendors that did not build it. Between October 27 and November 2, I ran that rule through eight more releases. The builders' mistakes were caught, as before. What I did not plan for was that the coordinator's mistakes got caught too. I count them as mine, because I run the coordinator.
The Setup
The coordinator is one Claude Opus 5.5 session. It plans each release, writes a short instruction for each builder, checks every review finding against the real code before passing it on, puts the pieces together, and ships. The builders are Claude, Codex running GPT-6.1 Sol, and Grok 4.7.
Think of a kitchen remodel with several crews. Each crew gets its own room, so two people are never tearing out the same wall. I call each room a lane: one slice of the release, with its own files, one builder, and two reviewers from the other vendors.
Earlier, in Five Reviews in One Release, I learned that a fix is new work and needs its own review. This post is about the step after that. The instruction that tells a builder what to fix is work too.
The last release is where it showed. It closed 21 issues in five lanes, and the full suite of 6,358 tests passed.
The Backup Check That Checked Nothing
Before that release, I ran an audit of email, memory and the AI models. First I made a backup of the app's data. The backup failed. The app had written two rows while the backup was being built, so the check at the end saw numbers that did not match. That became one of the 12 problems the audit found.
The backup fix went to Codex. Claude reviewed it and found that the new check held the database locked for the whole backup. Grok agreed and rated it high: if a scheduled backup was running, a chat message I sent at that moment could be lost.
So the coordinator wrote a new instruction for the builder: check the backup against "the rows the build read." It sounds reasonable. But the backup is made from the rows the build read. The check now compared the backup with itself, so it could never fail. A check that cannot fail is not a check.
Claude's next review caught it. The third design counts each table on its own, before and after the backup is built, and compares those counts. No lock, and a real missing row still fails.
The bug was not in the builder's code. It was in the coordinator's instruction.
Two More Wrong Instructions: Privacy and Unsubscribe
Privacy. The app has a local-only setting that keeps sensitive messages from going to a cloud AI model. Earlier in the week, fixing a different part of the app taught me a rule: a new version of a check may never be looser than the one already shipped. Whatever the old check caught, the new one must still catch. Then, in this fix, the coordinator's own round-two instruction made local-only looser than the code already shipped for one kind of message. Codex caught it. The old check now runs first, and the new one can only add to it.
So the coordinator broke a rule the coordinator had written. That is the reason I want someone else reading the instructions.
The unsubscribe scan. A reviewer flagged that "unsubscribe me from Netflix" made the app scan my Gmail. The coordinator's fix instruction removed that scan. But the issue's own acceptance criteria said "unsubscribe me from Medium" must scan, so the fix removed a scan the issue required. The next review pointed this out, and the coordinator put the scan back.
It was not only that release. In the pending-approvals release, one line in the coordinator's brief created a new way to approve everything at once. A Claude re-review caught it, and the coordinator reverted it. Earlier in the week, a Grok lane added a bare "approve" voice command nobody asked for. The premise in the coordinator's brief was wrong, and the lane "fixed" a problem that did not exist.
A Finding Is a Claim
The coordinator is not the only one who can be wrong. Once, a reviewer said some new test helpers were broken. The coordinator did not argue, and it did not send the builder back for another round. It ran the 362 tests that use those helpers, and every one passed. That settled it in minutes.
A review finding is a claim to check, not an order. That goes both ways: the reviewers check the coordinator, and the coordinator checks the reviewers.
What Works
- Reviewers checked the coordinator's instructions, not only the builders' code. I did not set it up for that, but it happened three times in one release.
- A written rule can be held to. "Never looser than what is already shipped" gives the coordinator a line to be held to, and the reviewers caught it crossing that line.
- Checking a finding before acting on it saves rounds. Running the tests settled one argument without another review.
What It Costs
- Rounds. Most lanes in the last release took two fix rounds. One took three. The backup check took three designs.
- The coordinator's time. More of it went to checking than building: checking findings, and checking its own instructions.
- Limits. In the last release, two Claude lanes hit a usage limit mid-round and waited for it to reset, while the other vendors kept working.
I did not measure what this cost in money, so this post makes no cost claims. Eight releases and one person judging them is not a benchmark, and I would not rank the models from it.
Where I Landed
I built this method so that nobody's work is trusted on their word alone. I assumed that meant the builders. It meant the coordinator too. An instruction is work, and it needs the same review as the code.
The notes for the next session now carry one more habit: before sending a fix round, check the instruction against the acceptance criteria and what is already shipped. The rule from my last post still holds. Every piece is checked by the two vendors that did not build it. I just did not expect "every piece" to include the coordinator's own instructions.
Work With Me
Running more than one AI agent on the same codebase? I help engineering teams set up coordinator and cross-vendor review workflows that catch what green tests miss, through MERA Consulting.
Comments