An agent can write a tutorial in seconds. It can write five hundred of them before lunch. The part that doesn’t get faster is you, sitting down to check whether any of it is actually correct.<\/p>\n\n\n\n
That’s the real bottleneck in AI-generated coding education, and it shows up fast if your product teaches step by step rather than dumping a finished solution on the student. A single Two Sum walkthrough might be eight or ten discrete steps \u2014 declare a hash map, open a loop, compute the complement, check membership, store the value, return the pair. Each step is a small, specific claim: given this code, doing this thing produces that code.<\/em> Multiply that by a real tutorial library and you’re not reviewing tutorials anymore, you’re reviewing thousands of tiny claims, one at a time, forever.<\/p>\n\n\n\n So the question worth answering properly is: can an agent produce that volume of tutorial content without a human reading every step? I think the answer is yes \u2014 but getting there requires giving up on the instinct that got you this far, which is making the agent more careful.<\/p>\n\n\n\n The natural response to a bad tutorial step is to patch the prompt. Add a rule. Then another. Six months in, the system prompt is a wall of “IMPORTANT: never do X” and the agent still occasionally does X, just less often.<\/p>\n\n\n\n This isn’t a prompting failure \u2014 it’s a category error. No software team ships reliable code because its engineers never make mistakes. They ship reliable code because the process<\/em> catches mistakes before they matter: compilers, type checkers, linters, test suites, CI, staged rollouts. The engineer is not the reliability mechanism. The pipeline is.<\/p>\n\n\n\n Agentic content generation needs the same shift. Stop trying to make the generator infallible. Build a system whose entire job is assuming the generator will eventually be wrong, and catching it when it is.<\/p>\n\n\n\n In the first pipeline, review time scales with output. In the second, it doesn’t \u2014 because most of the checking happens before a human is ever in the loop, and the human’s job shifts from reading artifacts<\/em> to maintaining the system that reads artifacts for them<\/em>.<\/p>\n\n\n\n Freeform markdown is hard to validate because nothing in it is addressable. “Somewhere in this 400-word explanation, is there a step that references a variable before it’s declared?” is a much harder question to answer reliably than it needs to be.<\/p>\n\n\n\n Ask the agent to return structured data instead:<\/p>\n\n\n\n Now every check has somewhere concrete to look. “Duplicate step ID,” “symbol referenced before it’s introduced,” “step requires a concept not yet taught” are no longer judgment calls \u2014 they’re a few lines of ordinary code walking a data structure. None of it needs a model in the loop:<\/p>\n\n\n\n The general rule: if plain code can check it reliably, don’t spend a model call on it. Save the model for the parts that are genuinely ambiguous.<\/p>\n\n\n\n Here’s the part that’s easy to get wrong in the other direction. Once you have a Don’t do that. What you want instead is a transformation contract<\/strong> \u2014 the small set of properties a correct answer to this step must satisfy, stated as requirements rather than as one specific implementation:<\/p>\n\n\n\n Multiple different pieces of code can satisfy that contract. That’s the point \u2014 you’re validating the idea, not the formatting, which also happens to be the right thing to teach on a platform whose whole pitch is problem-solving over syntax memorization.<\/p>\n\n\n\n The worst possible validator is one giant prompt: review this tutorial for correctness, clarity, and pedagogy, score it 1\u201310.<\/em> An 8\/10 tells you almost nothing. What failed? Can it be auto-fixed? Did the next version actually improve, or did the score just move around randomly?<\/p>\n\n\n\n Split it into atomic checks that each answer one specific yes\/no question:<\/p>\n\n\n\n Each one returns pass\/fail with evidence, not a vibe:<\/p>\n\n\n\n This is the difference between “48 passed, 2 failed, here’s exactly which and why” and “seems pretty good.” Only the first one is something you or an agent can act on.<\/p>\n\n\n\n Coding tutorials have an advantage almost no other AI-generated content type has: the artifact can be run. Use that aggressively \u2014 hidden test cases, generated edge cases, and for anything with multiple valid answers, property checks instead of exact-match checks:<\/p>\n\n\n\n The checker doesn’t need to know the<\/em> answer. It needs to know what must be true of any<\/em> acceptable answer. That single shift \u2014 from “does this match what I expected” to “does this satisfy what must hold” \u2014 is the most reusable idea in this whole approach, and it applies well beyond code: test cases, complexity claims, even prose explanations can often be reduced to a checkable property instead of a fixed target string.<\/p>\n\n\n\n This is the check that finds problems the others miss, and it’s worth building even though it’s the most expensive one on the list.<\/p>\n\n\n\n Give a second agent only the student’s current code and the instruction \u2014 not the expected answer. Ask it to perform the step. Compare what it produces against the transformation contract, not against the “official” solution.<\/p>\n\n\n\n If the instruction says “loop through This generalizes past tutorials: if an agent writes install instructions, run them in a clean container. If it writes API docs, hand them to an agent that’s never seen the implementation and have it integrate against the docs alone. Execution \u2014 literal or simulated \u2014 finds what review misses, because review already knows the answer.<\/p>\n\n\n\n Some things really do need judgment: is this step cognitively too big, does the explanation build the right intuition, does it match your house voice. That’s a legitimate job for a model. Two things make it work instead of just adding noise:<\/p>\n\n\n\n Keep the rubric narrow and specific.<\/strong> “Score this 1\u201310” produces a number you can’t act on. “Given that the student knows variables, arrays, and for-loops but not Maps or Sets \u2014 does this instruction require a concept outside that list?” produces a checkable yes\/no with evidence attached.<\/p>\n\n\n\n Don’t let one uncalibrated judge be a hard gate.<\/strong> A model judging model-generated content shares blind spots with the generator more often than you’d like \u2014 same training data, same habits, same things it doesn’t think to question. Run more than one judge where it matters, and pay attention when they disagree with each other; disagreement is a stronger signal than any individual score. Before trusting a judge’s rejections at all, measure how often it’s right against content you’ve already hand-checked.<\/p>\n\n\n\n Once checks are atomic, repair gets a lot easier, because you’re not asking the agent to guess what’s wrong.<\/p>\n\n\n\n Bad repair prompt: “there are some issues, please improve the tutorial.”<\/em> The agent will rewrite things that were fine and probably introduce something new while it’s at it.<\/p>\n\n\n\n Good repair prompt: exact failing checks, nothing else. “Fix only step It’s tempting to run one long session that generates, checks, and fixes fifty tutorials in a row, because the agent “has all the context.” What you actually get, a few hours in, is a session full of dozens of file reads, several abandoned approaches, and reasoning about a bug you already fixed three tutorials ago \u2014 context that’s technically present and practically useless.<\/p>\n\n\n\n Claude Code’s subagents exist for exactly this: a subagent runs in its own context window with its own focused prompt, does the noisy work (running the full test suite, executing edge cases, producing a verbose trace), and returns only the summary to the main thread. Use one for the deterministic-check pass, a separate one with a distinctly different persona for the judge pass, and keep intermediate results as files on disk \u2014 A flat “review 5% of everything” policy under-checks your riskiest content and over-checks your safest. A brand-new topic your platform has never taught before deserves a full read. The two-hundredth “two-pointer” tutorial in a pattern you’ve already validated a hundred times doesn’t need the same scrutiny.<\/p>\n\n\n\n Combine what you already have into a single risk score \u2014 deterministic failures, judge scores, how much your judges disagreed, how many repair attempts it took, whether the topic is new, and the historical defect rate for that specific tutorial template \u2014 and route on that:<\/p>\n\n\n\n That last line matters. Auto-approval without any ongoing audit is how a blind spot in the harness itself goes undetected for months.<\/p>\n\n\n\n Before this pipeline touches real content at volume, build a small, deliberately-broken test set. Take twenty or thirty solid tutorials and hand-seed specific defects into copies of them: an ambiguous instruction, a leaked future step, a concept used before it’s taught, a complexity claim that doesn’t match the actual runtime, a step that quietly modifies unrelated code.<\/p>\n\n\n\n Run the full pipeline against the set. What you care about most is what it misses<\/em> \u2014 a seeded bug your harness approves is a false negative, and false negatives are the failure mode that costs you a customer’s trust silently, long before you notice a pattern. Every time you change a prompt, swap a model, or add a new check, rerun this set. It’s a regression suite for a system that would otherwise have none.<\/p>\n\n\n\n When an auditor \u2014 or worse, a student \u2014 finds a real problem that made it through, the least useful response is fixing that one tutorial. The useful response is asking why the harness didn’t catch it, and turning the answer into a new atomic check, a new entry in the seeded-defect set, or a new blind-attempt scenario for the simulated-student pass. This is the actual compounding mechanism in the whole system: every escaped mistake becomes a permanent improvement instead of a one-off fire drill. Track it directly \u2014 what fraction of human-discovered defect categories got converted into a reusable, automated check<\/em> \u2014 because if the same class of error keeps reaching a human, the harness isn’t learning, no matter how good it looks on paper.<\/p>\n\n\n\n Raw tutorial count stopped being interesting the moment generation got cheap. What’s worth watching instead:<\/p>\n\n\n\n The pieces map fairly directly onto primitives already in the tool, and it’s worth using the right one for each job rather than pushing everything into one long prompt:<\/p>\n\n\n\n None of this is exotic. It’s a CI\/CD pipeline, aimed at content instead of application code, built on the same premise: assume failure, catch it cheaply and early, and reserve expensive human judgment for what’s actually left over.<\/p>\n\n\n\n Full autonomy \u2014 one prompt, ten thousand tutorials, zero verification \u2014 isn’t something worth wanting even if it were reliable, because the failures that survive a good harness are precisely the ones that need a human: is this the right thing to teach at this point in the curriculum, does this explanation build real understanding or just look like it does, is this genuinely a new kind of problem the harness has no precedent for.<\/p>\n\n\n\n What’s achievable, and worth building toward, is a system where the boring 90% \u2014 does it run, does it pass the tests, is the instruction internally consistent, does it introduce concepts in the right order \u2014 never touches a human at all, and your attention goes entirely to the fraction that’s actually about judgment. That’s not a weaker version of “agents you can trust.” For content with executable ground truth, it’s close to the real thing: not an agent that never makes mistakes, but a system built on the assumption that it will, and designed to catch them before anyone downstream has to.<\/p>\n","protected":false},"excerpt":{"rendered":" An agent can write a tutorial in seconds. It can write five hundred of them before lunch. The part that…<\/p>\n","protected":false},"author":1,"featured_media":8661,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[23],"tags":[],"class_list":["post-8657","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-problem-solving"],"_links":{"self":[{"href":"https:\/\/algocademy.com\/blog\/wp-json\/wp\/v2\/posts\/8657"}],"collection":[{"href":"https:\/\/algocademy.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/algocademy.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/algocademy.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/algocademy.com\/blog\/wp-json\/wp\/v2\/comments?post=8657"}],"version-history":[{"count":1,"href":"https:\/\/algocademy.com\/blog\/wp-json\/wp\/v2\/posts\/8657\/revisions"}],"predecessor-version":[{"id":8659,"href":"https:\/\/algocademy.com\/blog\/wp-json\/wp\/v2\/posts\/8657\/revisions\/8659"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/algocademy.com\/blog\/wp-json\/wp\/v2\/media\/8661"}],"wp:attachment":[{"href":"https:\/\/algocademy.com\/blog\/wp-json\/wp\/v2\/media?parent=8657"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/algocademy.com\/blog\/wp-json\/wp\/v2\/categories?post=8657"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/algocademy.com\/blog\/wp-json\/wp\/v2\/tags?post=8657"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}Careful generation is not the same as reliable output<\/h2>\n\n\n\n
naive: generate \u2192 human reads everything \u2192 publish\nwhat works: generate \u2192 automated checks \u2192 automated repair \u2192 risk score \u2192 publish or escalate\n<\/code><\/pre>\n\n\n\nGive the agent a contract, not a blank page<\/h2>\n\n\n\n
{\n \"problemId\": \"two-sum\",\n \"language\": \"javascript\",\n \"knownConcepts\": [\"variables\", \"arrays\", \"for-loops\"],\n \"steps\": [\n {\n \"id\": \"declare-map\",\n \"instruction\": \"Create a Map called `seen` to track numbers you've already visited.\",\n \"previousCode\": \"function twoSum(nums, target) {\\n\\n}\",\n \"expectedCode\": \"function twoSum(nums, target) {\\n const seen = new Map();\\n}\",\n \"introducesSymbols\": [\"seen\"],\n \"requiresConcepts\": [\"hash-map\"]\n }\n ]\n}\n<\/code><\/pre>\n\n\n\nconst ids = tutorial.steps.map(s => s.id);\nif (new Set(ids).size !== ids.length) throw new Error(\"Duplicate step ID\");\n<\/code><\/pre>\n\n\n\nTreat each step as a state transition, and validate the transition \u2014 loosely<\/h2>\n\n\n\n
previousCode<\/code> \/ expectedCode<\/code> pair for every step, the tempting move is exact comparison: does the student’s code match the expected code, character for character, or AST node for AST node?<\/p>\n\n\n\nfor (let i = 0; ...)<\/code> and for (let index = 0; ...)<\/code> are different ASTs and equally correct code. complement<\/code> and needed<\/code> are different identifiers holding the same idea. If your validator demands exact structural equality, you’ll reject good answers constantly and end up quietly retraining yourself to trust the tutorial less than the student’s actual reasoning.<\/p>\n\n\n\n{\n \"stepId\": \"index-loop\",\n \"requires\": [\"LOOP_ADDED\", \"TRAVERSES_NUMS\", \"CURRENT_INDEX_EXPOSED\"],\n \"forbids\": [\"LOOKUP_ADDED\", \"RETURN_ADDED\"]\n}\n<\/code><\/pre>\n\n\n\nBreak “is this any good” into small, boring questions<\/h2>\n\n\n\n
\n
{ \"check\": \"future-step-leakage\", \"stepId\": \"compute-complement\",\n \"pass\": false, \"evidence\": \"Explanation references seen.has(), introduced in the next step.\" }\n<\/code><\/pre>\n\n\n\nExecute the code. Check properties, not exact output.<\/h2>\n\n\n\n
function isValidTwoSum(nums: number[], target: number, result: number[]): boolean {\n const [i, j] = result;\n if (result.length !== 2 || i === j) return false;\n if (i < 0 || j < 0 || i >= nums.length || j >= nums.length) return false;\n return nums[i] + nums[j] === target;\n}\n<\/code><\/pre>\n\n\n\nLet an agent attempt the step blind \u2014 this catches what scoring can’t<\/h2>\n\n\n\n
nums<\/code>” and the contract requires the current index to be exposed, but the blind agent writes for (const num of nums)<\/code>, that’s not the agent being dumb \u2014 that’s the instruction failing to say what it needed to say. A human skimming the step, or an LLM asked “is this clear?”, will very often nod it through, because both are reading it already knowing the intended answer. An agent that has to act<\/em> on the instruction with no such knowledge exposes ambiguity that reading can’t.<\/p>\n\n\n\nresult = { outcome: \"AMBIGUOUS_INSTRUCTION\",\n required: \"CURRENT_INDEX_EXPOSED\",\n studentApproach: \"value-iteration, no index\" }\n<\/code><\/pre>\n\n\n\nUse model judges for the genuinely ambiguous parts \u2014 and don’t trust them blindly<\/h2>\n\n\n\n
Bound the repair loop \u2014 don’t let it run forever<\/h2>\n\n\n\n
compute-complement<\/code>, failing check future-step-leakage<\/code>, because the explanation names a method introduced in a later step. Do not modify passing steps.”<\/em> Rerun the full check suite afterward. Cap the number of attempts \u2014 three failed repair passes should escalate to a human, not trigger a fourth. An agent stuck in an unbounded fix-and-recheck loop isn’t autonomy, it’s just an expensive while(true)<\/code>.<\/p>\n\n\n\nStop letting one growing conversation hold everything<\/h2>\n\n\n\n
spec.json<\/code>, generated.json<\/code>, checks.json<\/code>, repair-1.json<\/code> \u2014 rather than as turns in a chat. The repair pass then only needs the failing checks and the artifact, not forty-five minutes of prior conversation. Treat context like a dependency you scope deliberately, not a transcript that accumulates by default.<\/p>\n\n\n\nRoute by risk, not by a flat sampling rate<\/h2>\n\n\n\n
risk < 0.10 \u2192 auto-publish\n0.10 \u2013 0.30 \u2192 extra automated pass, then publish\nrisk > 0.30 \u2192 human review\n always \u2192 audit a slice of what auto-published anyway\n<\/code><\/pre>\n\n\n\nDon’t trust the harness until it’s caught known bugs on purpose<\/h2>\n\n\n\n
Make every human-caught mistake permanent<\/h2>\n\n\n\n
Metrics that actually mean something<\/h2>\n\n\n\n
\n
Wiring this into Claude Code specifically<\/h2>\n\n\n\n
\n
claude -p<\/code>)<\/strong> is the right entry point for scripted, CI-style batch runs \u2014 generate a tutorial, pipe it through your check scripts, capture --output-format json<\/code>, branch on the exit code. It’s built for exactly this: no human in the loop, structured input and output, restricted tools via --allowedTools<\/code>.<\/li>\n\n\n\nPostToolUse<\/code> after a write, or PreToolUse<\/code> before anything risky \u2014 give you enforcement that can’t be skipped, because they’re deterministic scripts, not instructions the model has to remember to follow. Reserve them for the checks you genuinely cannot afford to have silently dropped.<\/li>\n\n\n\n-p<\/code> behavior available as a programmatic building block instead of a terminal command.<\/li>\n<\/ul>\n\n\n\nWhere this actually lands<\/h2>\n\n\n\n