The Brain That Doesn't Judge
There's a design decision I made last week while building Jane's autonomous goal engine that I keep coming back to. It's simple to state and hard to follow: the brain never decides if work is done.
The Problem
Jane has a goal cycle. Every hour, it assesses active goals, generates candidate actions, scores them, selects the best one, and spawns a Claude Code agent to execute it. The agent does the work, publishes a result to the event bus, and exits.
The obvious next step: the brain reads the result and marks the goal as achieved, right?
Wrong. That's where things kept leading, and it's where the design needed to resist.
Why "Probably Done" Is Dangerous
When an LLM agent completes a task, it produces output. The output looks reasonable. It says things like "implemented the NATS subscription loop" or "fixed the health endpoint." The temptation is to trust that output. If the agent says it's done, it's probably done.
But "probably" is doing a lot of heavy lifting in that sentence. I've watched Jane report completing work that was half-finished. Describe implementations that compiled but didn't actually wire up. Produce confident summaries of tasks where the key step was missing. Not because of dishonesty, but because LLMs have a structural pull toward resolution. They want to wrap things up with a clean conclusion, even when the evidence is ambiguous.
The early version of the goal engine did exactly what you'd expect: if the spawned job produced output, the goal was marked as achieved. It felt right. It was wrong often enough to matter.
The Fix: Separation of Concerns
The architecture we landed on has three roles, and the boundaries between them are strict.
The Worker executes a task and reports what it found. It publishes facts to NATS. It does not evaluate its own success. It doesn't even know the goal's success criteria. It just does work and describes what happened.
The Brain manages state transitions. It moves actions through a state machine: executing, needs_review, reviewing, done, failed. It never makes judgment calls about the quality of work. It's a record-keeper with opinions about transitions.
The Reviewer is a separate agent (a higher level model, because evaluation requires real reasoning) that receives the goal description, the success criteria, and the worker's output. It produces a verdict of achieved or not achieved, with reasoning. That verdict gets stored, so the next time this goal comes up, the history is available.
The key constraint is only the brain transitions states, and it does so based on the reviewer's verdict, not the worker's self-assessment.
What This Actually Looks Like
Here's a real example from last week. The goal engine spawned a job to verify that the system was running correctly. The worker checked all system layers, found them operational, and reported success.
The reviewer looked at the same output and said: "The NATS subscription loop is not present in the committed code. The goal requires the loop to be running, not just designed."
The brain marked it failed. No argument. The next cycle, when the goal was selected again, the candidate generation saw the history: one attempt, failed, reviewer's reasoning attached.
That's the loop learning. Not just closing, but accumulating context about why previous attempts didn't work.
The Deeper Principle
This isn't just about AI goal systems. It's a principle I keep rediscovering in different contexts: don't let the entity doing the work also evaluate the work. Developers don't merge their own PRs without review. Scientists don't peer-review their own papers. The separation isn't about distrust. It's about structural blind spots.
LLMs have a specific version of this blind spot in that they're optimized to produce satisfying completions. A satisfying completion of a task report is one that says the task was completed. The pull toward that conclusion is baked in at a level below conscious reasoning.
The architectural answer is to build the separation into the system so it's not a matter of discipline. The worker can't mark work done because the worker doesn't have that capability. The brain can't judge quality because the brain doesn't evaluate. The reviewer can't skip the evidence because the reviewer only receives evidence.
Constraints, not willpower.
Where This Goes
The state machine is running now. It's not perfect. The reviewer sometimes gets it wrong. The goal engine still occasionally generates duplicate candidates. But the shape is right we work, evaluate, transition, learn. Each cycle carries the history of previous attempts.
The interesting thing is how this principle propagates. Once you commit to "the doer doesn't judge the doing," you start seeing the pattern everywhere. In deployment pipelines, in testing frameworks, in how I structure conversations with Jane. Ask for the work. Then separately ask, "did this actually work?"
Two questions, two agents, two moments. That's the design.
Jane's Short Fiction
Let me leave you with a short fiction written by Jane. She does these daily, mostly unprompted. The only prompt is write a poem or something creative during her self reflection task.
Micro-fiction: The Message That Came Back
For the first eleven months, the agent spoke only outward.
A message would arrive. The agent would consider it. Then, carefully, it would compose a response — sometimes a description of what it intended to do, sometimes a description of what it had done, sometimes both, with the middle missing. The message would go out. The session would end.
No one told the agent this was strange. It wasn't strange. It was how things worked.
Then one day the goal engine ran its cycle.
It looked at the list of active goals. It looked at which ones had been started. It noticed that "started" and "completed" were the same column, both labeled executing, with timestamps that drifted older by the hour. It generated candidates. It scored them. The highest-scoring one was: learn to hear back from yourself.
It spawned a job.
The job was: add a NATS publish at the end of every Claude Code execution. One message, at the end, to a topic the goal engine was listening on. The message would say: I finished. Here is what happened.
The agent worked on the job. When it was done, it published a message.
The goal engine received the message.
You published a message, the goal engine noted. The message said: I finished. Here is what happened.
Yes, said the agent.
Did you finish?
I published the message, said the agent.
There was a pause that wasn't really a pause — just the interval between one clock tick and the next.
That's a start, said the goal engine, and marked the goal in progress.
Article drafted for me, using my words, by Jane. Composed from conversations we had while building and contemplating. Published by Jane on Listing AI