An agent can write a tutorial in seconds. It can write five hundred of them before lunch. The part that doesn’t get faster is you, sitting down to check whether any of it is actually correct.

That’s the real bottleneck in AI-generated coding education, and it shows up fast if your product teaches step by step rather than dumping a finished solution on the student. A single Two Sum walkthrough might be eight or ten discrete steps — declare a hash map, open a loop, compute the complement, check membership, store the value, return the pair. Each step is a small, specific claim: given this code, doing this thing produces that code. Multiply that by a real tutorial library and you’re not reviewing tutorials anymore, you’re reviewing thousands of tiny claims, one at a time, forever.

So the question worth answering properly is: can an agent produce that volume of tutorial content without a human reading every step? I think the answer is yes — but getting there requires giving up on the instinct that got you this far, which is making the agent more careful.

Careful generation is not the same as reliable output

The natural response to a bad tutorial step is to patch the prompt. Add a rule. Then another. Six months in, the system prompt is a wall of “IMPORTANT: never do X” and the agent still occasionally does X, just less often.

This isn’t a prompting failure — it’s a category error. No software team ships reliable code because its engineers never make mistakes. They ship reliable code because the process catches mistakes before they matter: compilers, type checkers, linters, test suites, CI, staged rollouts. The engineer is not the reliability mechanism. The pipeline is.

Agentic content generation needs the same shift. Stop trying to make the generator infallible. Build a system whose entire job is assuming the generator will eventually be wrong, and catching it when it is.

naive:      generate → human reads everything → publish
what works: generate → automated checks → automated repair → risk score → publish or escalate

In the first pipeline, review time scales with output. In the second, it doesn’t — because most of the checking happens before a human is ever in the loop, and the human’s job shifts from reading artifacts to maintaining the system that reads artifacts for them.

Give the agent a contract, not a blank page

Freeform markdown is hard to validate because nothing in it is addressable. “Somewhere in this 400-word explanation, is there a step that references a variable before it’s declared?” is a much harder question to answer reliably than it needs to be.

Ask the agent to return structured data instead:

{
  "problemId": "two-sum",
  "language": "javascript",
  "knownConcepts": ["variables", "arrays", "for-loops"],
  "steps": [
    {
      "id": "declare-map",
      "instruction": "Create a Map called `seen` to track numbers you've already visited.",
      "previousCode": "function twoSum(nums, target) {\n\n}",
      "expectedCode": "function twoSum(nums, target) {\n  const seen = new Map();\n}",
      "introducesSymbols": ["seen"],
      "requiresConcepts": ["hash-map"]
    }
  ]
}

Now every check has somewhere concrete to look. “Duplicate step ID,” “symbol referenced before it’s introduced,” “step requires a concept not yet taught” are no longer judgment calls — they’re a few lines of ordinary code walking a data structure. None of it needs a model in the loop:

const ids = tutorial.steps.map(s => s.id);
if (new Set(ids).size !== ids.length) throw new Error("Duplicate step ID");

The general rule: if plain code can check it reliably, don’t spend a model call on it. Save the model for the parts that are genuinely ambiguous.

Treat each step as a state transition, and validate the transition — loosely

Here’s the part that’s easy to get wrong in the other direction. Once you have a previousCode / expectedCode pair for every step, the tempting move is exact comparison: does the student’s code match the expected code, character for character, or AST node for AST node?

Don’t do that. for (let i = 0; ...) and for (let index = 0; ...) are different ASTs and equally correct code. complement and needed are different identifiers holding the same idea. If your validator demands exact structural equality, you’ll reject good answers constantly and end up quietly retraining yourself to trust the tutorial less than the student’s actual reasoning.

What you want instead is a transformation contract — the small set of properties a correct answer to this step must satisfy, stated as requirements rather than as one specific implementation:

{
  "stepId": "index-loop",
  "requires": ["LOOP_ADDED", "TRAVERSES_NUMS", "CURRENT_INDEX_EXPOSED"],
  "forbids": ["LOOKUP_ADDED", "RETURN_ADDED"]
}

Multiple different pieces of code can satisfy that contract. That’s the point — you’re validating the idea, not the formatting, which also happens to be the right thing to teach on a platform whose whole pitch is problem-solving over syntax memorization.

Break “is this any good” into small, boring questions

The worst possible validator is one giant prompt: review this tutorial for correctness, clarity, and pedagogy, score it 1–10. An 8/10 tells you almost nothing. What failed? Can it be auto-fixed? Did the next version actually improve, or did the score just move around randomly?

Split it into atomic checks that each answer one specific yes/no question:

Each one returns pass/fail with evidence, not a vibe:

{ "check": "future-step-leakage", "stepId": "compute-complement",
  "pass": false, "evidence": "Explanation references seen.has(), introduced in the next step." }

This is the difference between “48 passed, 2 failed, here’s exactly which and why” and “seems pretty good.” Only the first one is something you or an agent can act on.

Execute the code. Check properties, not exact output.

Coding tutorials have an advantage almost no other AI-generated content type has: the artifact can be run. Use that aggressively — hidden test cases, generated edge cases, and for anything with multiple valid answers, property checks instead of exact-match checks:

function isValidTwoSum(nums: number[], target: number, result: number[]): boolean {
  const [i, j] = result;
  if (result.length !== 2 || i === j) return false;
  if (i < 0 || j < 0 || i >= nums.length || j >= nums.length) return false;
  return nums[i] + nums[j] === target;
}

The checker doesn’t need to know the answer. It needs to know what must be true of any acceptable answer. That single shift — from “does this match what I expected” to “does this satisfy what must hold” — is the most reusable idea in this whole approach, and it applies well beyond code: test cases, complexity claims, even prose explanations can often be reduced to a checkable property instead of a fixed target string.

Let an agent attempt the step blind — this catches what scoring can’t

This is the check that finds problems the others miss, and it’s worth building even though it’s the most expensive one on the list.

Give a second agent only the student’s current code and the instruction — not the expected answer. Ask it to perform the step. Compare what it produces against the transformation contract, not against the “official” solution.

If the instruction says “loop through nums” and the contract requires the current index to be exposed, but the blind agent writes for (const num of nums), that’s not the agent being dumb — that’s the instruction failing to say what it needed to say. A human skimming the step, or an LLM asked “is this clear?”, will very often nod it through, because both are reading it already knowing the intended answer. An agent that has to act on the instruction with no such knowledge exposes ambiguity that reading can’t.

result = { outcome: "AMBIGUOUS_INSTRUCTION",
           required: "CURRENT_INDEX_EXPOSED",
           studentApproach: "value-iteration, no index" }

This generalizes past tutorials: if an agent writes install instructions, run them in a clean container. If it writes API docs, hand them to an agent that’s never seen the implementation and have it integrate against the docs alone. Execution — literal or simulated — finds what review misses, because review already knows the answer.

Use model judges for the genuinely ambiguous parts — and don’t trust them blindly

Some things really do need judgment: is this step cognitively too big, does the explanation build the right intuition, does it match your house voice. That’s a legitimate job for a model. Two things make it work instead of just adding noise:

Keep the rubric narrow and specific. “Score this 1–10” produces a number you can’t act on. “Given that the student knows variables, arrays, and for-loops but not Maps or Sets — does this instruction require a concept outside that list?” produces a checkable yes/no with evidence attached.

Don’t let one uncalibrated judge be a hard gate. A model judging model-generated content shares blind spots with the generator more often than you’d like — same training data, same habits, same things it doesn’t think to question. Run more than one judge where it matters, and pay attention when they disagree with each other; disagreement is a stronger signal than any individual score. Before trusting a judge’s rejections at all, measure how often it’s right against content you’ve already hand-checked.

Bound the repair loop — don’t let it run forever

Once checks are atomic, repair gets a lot easier, because you’re not asking the agent to guess what’s wrong.

Bad repair prompt: “there are some issues, please improve the tutorial.” The agent will rewrite things that were fine and probably introduce something new while it’s at it.

Good repair prompt: exact failing checks, nothing else. “Fix only step compute-complement, failing check future-step-leakage, because the explanation names a method introduced in a later step. Do not modify passing steps.” Rerun the full check suite afterward. Cap the number of attempts — three failed repair passes should escalate to a human, not trigger a fourth. An agent stuck in an unbounded fix-and-recheck loop isn’t autonomy, it’s just an expensive while(true).

Stop letting one growing conversation hold everything

It’s tempting to run one long session that generates, checks, and fixes fifty tutorials in a row, because the agent “has all the context.” What you actually get, a few hours in, is a session full of dozens of file reads, several abandoned approaches, and reasoning about a bug you already fixed three tutorials ago — context that’s technically present and practically useless.

Claude Code’s subagents exist for exactly this: a subagent runs in its own context window with its own focused prompt, does the noisy work (running the full test suite, executing edge cases, producing a verbose trace), and returns only the summary to the main thread. Use one for the deterministic-check pass, a separate one with a distinctly different persona for the judge pass, and keep intermediate results as files on disk — spec.json, generated.json, checks.json, repair-1.json — rather than as turns in a chat. The repair pass then only needs the failing checks and the artifact, not forty-five minutes of prior conversation. Treat context like a dependency you scope deliberately, not a transcript that accumulates by default.

Route by risk, not by a flat sampling rate

A flat “review 5% of everything” policy under-checks your riskiest content and over-checks your safest. A brand-new topic your platform has never taught before deserves a full read. The two-hundredth “two-pointer” tutorial in a pattern you’ve already validated a hundred times doesn’t need the same scrutiny.

Combine what you already have into a single risk score — deterministic failures, judge scores, how much your judges disagreed, how many repair attempts it took, whether the topic is new, and the historical defect rate for that specific tutorial template — and route on that:

risk < 0.10   → auto-publish
0.10 – 0.30   → extra automated pass, then publish
risk > 0.30   → human review
     always   → audit a slice of what auto-published anyway

That last line matters. Auto-approval without any ongoing audit is how a blind spot in the harness itself goes undetected for months.

Don’t trust the harness until it’s caught known bugs on purpose

Before this pipeline touches real content at volume, build a small, deliberately-broken test set. Take twenty or thirty solid tutorials and hand-seed specific defects into copies of them: an ambiguous instruction, a leaked future step, a concept used before it’s taught, a complexity claim that doesn’t match the actual runtime, a step that quietly modifies unrelated code.

Run the full pipeline against the set. What you care about most is what it misses — a seeded bug your harness approves is a false negative, and false negatives are the failure mode that costs you a customer’s trust silently, long before you notice a pattern. Every time you change a prompt, swap a model, or add a new check, rerun this set. It’s a regression suite for a system that would otherwise have none.

Make every human-caught mistake permanent

When an auditor — or worse, a student — finds a real problem that made it through, the least useful response is fixing that one tutorial. The useful response is asking why the harness didn’t catch it, and turning the answer into a new atomic check, a new entry in the seeded-defect set, or a new blind-attempt scenario for the simulated-student pass. This is the actual compounding mechanism in the whole system: every escaped mistake becomes a permanent improvement instead of a one-off fire drill. Track it directly — what fraction of human-discovered defect categories got converted into a reusable, automated check — because if the same class of error keeps reaching a human, the harness isn’t learning, no matter how good it looks on paper.

Metrics that actually mean something

Raw tutorial count stopped being interesting the moment generation got cheap. What’s worth watching instead:

Wiring this into Claude Code specifically

The pieces map fairly directly onto primitives already in the tool, and it’s worth using the right one for each job rather than pushing everything into one long prompt:

None of this is exotic. It’s a CI/CD pipeline, aimed at content instead of application code, built on the same premise: assume failure, catch it cheaply and early, and reserve expensive human judgment for what’s actually left over.

Where this actually lands

Full autonomy — one prompt, ten thousand tutorials, zero verification — isn’t something worth wanting even if it were reliable, because the failures that survive a good harness are precisely the ones that need a human: is this the right thing to teach at this point in the curriculum, does this explanation build real understanding or just look like it does, is this genuinely a new kind of problem the harness has no precedent for.

What’s achievable, and worth building toward, is a system where the boring 90% — does it run, does it pass the tests, is the instruction internally consistent, does it introduce concepts in the right order — never touches a human at all, and your attention goes entirely to the fraction that’s actually about judgment. That’s not a weaker version of “agents you can trust.” For content with executable ground truth, it’s close to the real thing: not an agent that never makes mistakes, but a system built on the assumption that it will, and designed to catch them before anyone downstream has to.