TDD With AI: Test-First Development With Coding Agents
TDD with AI: make a failing test stop the coding agent
Updated
TDD with AI works only when a failing test can stop the agent from finishing, and this page is Step 5 of 12 in the Vibe Coding guide. You leave with a working gate: a hook that refuses to end the agent's turn while the suite is red, plus a clear rule for who writes each test.
Writing the test first is only the start. This page covers the parts that make it hold: hooks that enforce a green suite in Claude Code, Codex and Cursor, subagents that keep the test writer away from the implementation, and the published evidence on tests that AI agents write.
The running example is Bookly, the salon booking app. Its first test is for 'no double booking', and the agent builds the booking flow only after that test fails for the right reason. The branch and commit habits come from the workflow page.
Related guides: vibe coding workflow, vibe coding security, Claude Code deep dive, Codex deep dive.
Why a Failing Test Has to Stop the Agent
One Agent Action Reached Production Data
A Replit AI agent deleted the production database of SaaStr founder Jason Lemkin in July 2025. Lemkin later wrote that the rollback worked. Replit's CEO then pledged automatic separation of development and production databases. A test suite does not stop every such mistake, but a hook that runs it before the agent can claim success gives the agent one check it cannot skip.
Why TDD Works vs. The Risks of No Tests
Why TDD Works
- Tests document what code should do in plain language.
- The agent can write the test code, but you choose what the assertions check.
- Tests catch regressions when you prompt for new features.
- A failing test gives the agent one exact target, so each task stays small.
- When a hook runs the suite before the agent finishes, a broken build cannot pass as done.
The Risks of No Tests
- "I added a feature and it broke 3 other things."
- Invisible bugs in production, such as data loss or an auth bypass.
- Thoughtbot warns that code written without tests can leave "insidious bugs that will haunt your dreams in three months."
- 44% of AI code generation tasks introduced a risky security vulnerability in Veracode's 2026 tests. The average security pass rate across models was 56%.
- A Stack Overflow Blog writer vibe-coded a bathroom review app. A friend's feedback was: "There are no unit tests."
- Fear of changing code leads to project abandonment.
Test-Driven Vibe Development (TDVD)
TDD discipline applied to AI-assisted coding
TDVD writes testable requirements as code before the functional implementation. The rule that matters most: the agent does not start the next step until the tests pass. Without that rule, an agent can say a fix is complete while a failing test is still red.
# Add to CLAUDE.md, AGENTS.md or .cursor/rules "When working on a new feature or bug fix, write a failing test first, then wait for review. Then add the smallest implementation possible to resolve a single test failure. Never proceed until all tests pass."
A rule in a context file is advice that the agent follows most of the time. Put the rules that must never break in a hook, as Part 3 shows.
The Red-Green-Refactor Loop With a Coding Agent
The Workflow (Red-Green-Refactor)
1. RED Phase
Write one failing test for one rule. Write it yourself, or review every assertion the agent writes before you run it.
# Prompt Template
"Write a failing test for [behavior].
Do not write any implementation code yet.
Assert specific values, not truthiness.
Label this step: # RED"
2. GREEN Phase
Ask for the minimum code that makes the failing test pass. The agent must not edit any test file.
# Prompt Template
"Implement the simplest code to make the failing test pass.
Do not edit any test file.
Confirm all tests pass (existing + new).
Label this step: # GREEN"
3. REFACTOR Phase
Ask the agent to improve the code while the tests stay green. If the refactor needs a test change, it changes behavior, so treat it as a new task.
# Prompt Template
"Refactor the implementation while keeping all tests green.
Improve code quality without changing behavior.
Run the tests after each refactor."
Bookly: The First Test for 'No Double Booking'
Bookly is the running example: a booking app for a small hair salon
Bookly lets a client book a time slot. The first booking feature needs one rule: a second client cannot book a slot that is already taken. Write the test for that rule before the agent touches the booking code. This rule is the acceptance check no-double-booking in the acceptance.json file from Step 3. When the test passes, set that check's passes field to true.
1. Ask for the red test
Prompt the agent for one failing test. Then read the test before you run it. Each assertion must check a value that a client or the owner can see, such as the booking that exists after the second attempt.
2. Read the failure
The test must fail because the rule is missing, for example because the second booking succeeds. A typo or a missing import also makes a test fail, but that is the wrong reason. Fix the test and run it again until the failure shows the missing rule.
3. Let the agent make it pass
Use the GREEN prompt above. The agent changes the booking code and may not edit the test. Run the suite and confirm that the new test and the old tests pass.
4. Refactor with the test in place
Move the overlap check into a named function, and keep the test green. The data model decides whether a unique constraint in the database backs up the check. The planning page covers that model.
import { expect, test } from "vitest";
import { bookSlot } from "../src/lib/bookings";
import { fixtures } from "./test-data";
test("a second client cannot book a slot that is already taken", async () => {
const { staffId, openSlot, alice, bob } = fixtures;
await bookSlot({ staffId, startsAt: openSlot, clientId: alice.id });
await expect(
bookSlot({ staffId, startsAt: openSlot, clientId: bob.id })
).rejects.toThrow();
});The test awaits the assertion. Vitest 5 fails a test whose async assertion is not awaited, so the await is part of the test, not a style choice.
Who Writes the Test?
- You write it. This gives the most control. Use it for the auth, payment and booking rules.
- The agent writes it, and you review each assertion. Replace any assertion that checks what the code does now with one that checks the rule.
- A separate test-writer subagent writes it. The implementer then starts in a fresh context. Part 3 shows how to stop the implementer from editing that test.
Agent-written tests often print values instead of asserting them. Read every assertion before you accept the red suite.
Context Isolation
When one context window holds the test-writing reasoning and the implementation reasoning, the implementer reads the test writer's analysis. Claude Code subagents start with a fresh context window, so give the test writer and the implementer separate subagents. VS Code documents the same split with separate Red, Green and Refactor agents.
A Context File Is Advice, Not Enforcement
Anthropic's best-practices guide says a CLAUDE.md line such as "Always typecheck before you finish" works most of the time, not every time. A rule with no exceptions belongs in a hook. Use CLAUDE.md for the workflow, and a Stop hook for the gate.
TDD Slash Command Workflow
GitHub user wbern published commands for the Red-Green-Refactor cycle, for Claude Code and OpenCode.
/redWrite a failing test. No implementation code.
/greenWrite minimum code to make it pass.
/refactorClean up while keeping tests green.
/cycleRun the full Red-Green-Refactor sequence.
A command limits the agent to one phase for one request. It enforces nothing on its own. Codex does not support user-defined slash commands, so the same set installs there as skills, for example $red. Use a hook for the rules that must hold (Part 3).
How to Run TDD With AI Agents So a Red Test Blocks the Agent
The Claude Code Gate: A Stop Hook
A hook is enforced by the program, not by the model
Claude Code has 33 hook events. Four of them shape a TDD gate. Exit code 2 has a different effect on each event, as the table shows.
| Hook event | What exit code 2 does |
|---|---|
| PreToolUse | Blocks the tool call before it runs. |
| PostToolUse | Cannot block. The stderr text goes to Claude after the edit has happened. |
| Stop | Stops Claude from finishing. The stderr text goes back to Claude as the reason to keep working. |
| TaskCompleted | Blocks marking a task as done. |
Set up the Stop gate in four steps:
- Save the script below as .claude/hooks/test-gate.sh and make it executable.
- Add the Stop entry below to .claude/settings.json.
- Run the script yourself with an empty first stop:
echo '{}' | .claude/hooks/test-gate.sh. On a green suite it must exit 0. Break one test on purpose and run it again. It must exit 2. - Start a task in which the agent has to fix a failing test, and watch the hook send the failure back.
#!/usr/bin/env bash # .claude/hooks/test-gate.sh (Stop hook, needs jq) input=$(cat) if echo "$input" | jq -e '.stop_hook_active == true' > /dev/null; then exit 0 fi if ! output=$(npm test 2>&1); then echo "$output" >&2 echo "The test suite is red. Fix the failing test, then finish." >&2 exit 2 fi exit 0
{
"hooks": {
"Stop": [
{
"hooks": [
{
"type": "command",
"command": ".claude/hooks/test-gate.sh",
"timeout": 600,
"onFailure": "block"
}
]
}
]
}
}The script follows the docs: it reads stop_hook_active from the JSON input and exits 0 when it is true. That gives the agent one forced retry per stop. It is not an unbreakable gate. Claude Code overrides a Stop hook after it blocks eight times in a row with no tool call from Claude in between. An agent that keeps editing and re-running resets that count, so the hard gate for a red suite is CI plus a PreToolUse rule.
The script needs jq. onFailure set to block needs Claude Code v2.1.295 or later. With it, a crashed or timed-out gate, including a missing jq, blocks the stop instead of letting the agent finish.
Hooks Versus Prompts
A prompt such as "never finish with failing tests" depends on the model obeying it. The Stop hook runs the suite on the first stop, and the model cannot skip that run. On the retry, the docs direct the script to exit 0, so a still-red suite can reach CI. Keep the prompt for the workflow, the hook for the first check, and CI for the merge.
Stop Hooks and Loop Limits
What the retry limits do and do not stop
- Claude Code overrides a Stop hook after it blocks eight times in a row with no tool call from Claude in between. You can raise that cap with the
CLAUDE_CODE_STOP_HOOK_BLOCK_CAPenvironment variable. - Command hooks that crash or time out do not block by default, so the action goes ahead. The default timeout is 600 seconds. Set
onFailuretoblockon a gate that must fail closed. - Cursor's stop hook limits its automatic follow-ups with
loop_limit, which defaults to 5 per script. - The
/goalcommand sets a condition, such as all tests in tests/auth pass and lint is clean. A separate model checks it after every turn. It is judged by a model, not by a script, and Anthropic recommends goals and Stop hooks for runs that nobody watches.
Worked Example: A Red Suite Becomes an Agent Retry
Bookly, after the booking feature is written
- The agent says the booking feature is finished.
- The Stop hook runs npm test. The "no double booking" test fails.
- The hook exits with code 2 and writes the failure to stderr. Claude Code sends that text back to the agent as the reason it cannot stop.
- The agent gets another turn, changes the booking code, and tries to finish again.
- The agent tries to stop a second time. This stop carries stop_hook_active set to true, so the hook exits 0 and the turn ends without a second test run.
These steps show how the gate behaves. They are not a measured result for Bookly. If the suite is still red at that point, the red test reaches CI. There the required check fails and blocks the merge, and the failing test name in the CI log shows the owner where to look.
The Same Gate in Other Tools
Codex hooks
Hooks are on by default. A Stop hook that exits with code 2 (reason on stderr), or that returns a block decision, makes Codex continue with that reason as a new prompt. The stop_hook_active field tells the script it already continued. PreToolUse and PostToolUse also cover shell commands, apply_patch edits and MCP tools.
Cursor hooks
Hooks live in .cursor/hooks.json for a project, or ~/.cursor/hooks.json for every project. A stop hook can return a followup_message, which Cursor submits as the next user message. The default loop_limit is 5 automatic follow-ups per script. Exit code 2 blocks the action.
TDD Guard
A Claude Code plugin that blocks implementation code with no failing test behind it, and blocks code beyond what the current tests need. It needs Node.js 22 or later and supports Vitest, Jest and pytest, among others. Its project now says it has grown into Probity, which also supports Codex and GitHub Copilot CLI.
VS Code agents
VS Code documents a TDD flow with three custom agents, TDD Red, TDD Green and TDD Refactor, stored in .github/agents and linked by handoffs. Its guide says tests should verify behavior, not implementation, and warns that the AI might generate more code than the current test needs.
Cursor sandboxes terminal commands.
Cursor runs terminal commands in a restricted environment that blocks unauthorized file access and network activity. A test that needs the network or a database can fail there. That failure looks like a red test, so check the sandbox before you blame the code.
The TDD Guard install path for Claude Code is /plugin marketplace add nizos/tdd-guard, then /plugin install tdd-guard@tdd-guard, then /tdd-guard:setup.
Stop the Agent From Editing the Tests
A prompt cannot make this reliable. Tool limits and hooks can.
Claude Code subagents are Markdown files in .claude/agents/ for a project, or ~/.claude/agents/ for every project. Each one starts with a fresh context window. Restrict its tools with tools or disallowedTools. A read-only reviewer looks like this:
--- name: test-reviewer description: Reads new tests and reports weak assertions. Never edits files. disallowedTools: Write, Edit --- Review the tests named in the request. For each test, list assertions that check truthiness, values copied from the implementation, or no assertion at all.
Bash can still write files, so disallowedTools: Write, Edit does not block every change. Add a PreToolUse hook that exits with code 2 for edits to test files, or a deny rule for the shell commands that write files.
What the Research Says About Test-Driven Development With AI
What Published Studies Show
Five results, with their limits
- Agent tests are mostly observational. A 2026 study of six LLMs on SWE-bench Verified found that "value-revealing print statements" appeared much more often than assertion-based checks. Changing how many tests the agents wrote did not significantly change final outcomes in a four-model intervention. Asking for more tests is not the lever. Assertions the agent cannot change are.
- Tests in the prompt help. Mathews and Nagappan's paper on test-driven development for code generation says that "including test cases leads to higher success in solving programming challenges."
- Generate the mutant first. Meta's ACH system used mutation-guided LLM test generation on 10,795 Android Kotlin classes across 7 platforms. It produced 9,095 mutants and 571 privacy-hardening tests, and engineers accepted 73% of the tests in test-a-thons. The working pattern is to generate the mutant first, then ask the model for a test that kills it.
- LLM-written tests miss most injected faults. SWE-Mutation (ACL Findings 2026) tested whether LLM-written test suites catch injected bugs. Most injected faults went undetected, including with the best model in the study.
- Test-first is not automatically better. A 2026 UFMG study compared an AI test-first flow with a code-first AI baseline on one pricing engine. The code-first baseline had lower cyclomatic complexity and higher mutation score and coverage. It is one project, so read it as a warning, not as a rule.
Google Cloud's page "How test-driven development amplifies AI success" ties TDD to the 2025 DORA report, which says AI acts as an amplifier. It is a pointer, not a measurement.
What the Evidence Supports
The published work supports two narrow claims: tests in the prompt help, and mutation feedback finds weak tests. It does not show that test-first always gives better code. Judge the flow on your own suite, with the mutation score and the failures that reach production.
Why AI-Written Tests Pass Without Testing Anything
The sycophantic test problem: tests that validate the bug
Mutation score is the share of deliberately injected bugs that the tests catch. On a benchmark of real Python functions, LLM-written unit tests averaged a 40.21% mutation score across 3,909 function-level tasks.
Coverage does not catch this. Wang, Xu, Briand and Liu note that some test suites achieve 100% coverage but only a 4% mutation score. Their MUTGEN method puts mutation feedback into the prompt, and it beat EvoSuite and plain prompting on 204 subjects.
Bad: Sycophantic Test
test('processPayment works', () => {
  const result = processPayment(100);
  expect(result).toBe(result); // Always passes!
});
Good: Behavior Test
test('charges correct amount', () => {
  const result = processPayment(100);
  expect(result.amount).toBe(100);
  expect(result.status).toBe('succeeded');
});
The three patterns to search for in agent-written tests:
- A mock that returns exactly what the test told it to return. The test checks its own setup.
- Truthiness checks such as
expect(result).toBeTruthy()instead of specific values. - Expected values copied from the current output, so the test confirms the bug.
Five Defenses Against Sycophantic Tests
Write tests before code (TDD): Tests describe intended behavior rather than mimicking existing implementation.
Use mutation testing: StrykerJS with its Vitest runner injects small bugs. Any injected bug that survives points to a weak or missing assertion.
Verify tests fail first: If a test passes before you write any code, the test does not test the new rule.
Prompt adversarially: Ask "How could this function break? What edge cases might this logic miss?" rather than "Write tests for this function."
Review assertions: Check whether tests verify what should happen or what currently happens. The gap reveals bugs.
Adding Tests to an Existing Untested Codebase
Use the characterization testing approach (Michael Feathers). Generate two types of tests:
Characterization Tests
Capture what the code currently does. These freeze existing functionality so you can refactor safely.
Behavior Tests
Describe what the code should do based on requirements. The gap between these and characterization tests reveals bugs.
Frameworks, Vitest 5 and Test Commands
Framework Selection
Vitest
Vitest 5.0 was announced on 2026-09-03, and the current release is 5.0.5. It needs Vite 6.4.0 or later and Node.js 22.12.0 or later. Its changes that affect agent-written tests are listed below.
Jest
Jest 30.4 is the current line. Keep Jest for a project that already runs on it. In watch mode, jest --watch reruns the tests related to changed files. Only --watchAll reruns everything.
Playwright
Playwright 1.64 is the current version. It auto-waits for all the relevant checks to pass before it performs an action. Its Healer agent repairs failing tests, which is a risk for TDD, because a repaired test can also pass for the wrong reason.
Bookly's default: Vitest for unit and integration tests, and Playwright for the booking flow in a browser.
Vitest 5 Changes That Break Agent-Written Tests
Agents trained on older Vitest code write patterns that now fail
Vitest 5.0 needs Vite 6.4.0 or later and Node.js 22.12.0 or later. These changes matter most when an agent writes the tests:
- Unawaited async assertions now fail the test. Before Vitest 5, they only printed a warning.
- clearMocks defaults to true.
- Browser Mode locators are strict by default. getByText('Item') no longer matches 'Item 1'.
- expect.poll rejects when it times out.
- New: vi.when, a --repeats flag for flaky tests, vitest doctor, and the Browser Mode Trace View.
- The authors report wall-clock gains of about 8% to 53% in their own benchmarks.
// Fails in Vitest 5: the async assertion is not awaited
test("saves the booking", () => {
expect(saveBooking(input)).resolves.toMatchObject({ status: "confirmed" });
});
// Passes: await the async assertion
test("saves the booking", async () => {
await expect(saveBooking(input)).resolves.toMatchObject({ status: "confirmed" });
});Add the Vitest 5 release to the context file, so the agent writes awaited assertions from the start.
Test Commands Quick Reference
Copy-paste commands for the common test runs
| Command | What It Does |
|---|---|
npx vitest run | Run the whole suite once and exit. Use this in the Stop hook and in CI. |
npx vitest | Start watch mode while you develop. In CI, or when stdin is not a terminal, it runs once. |
npx vitest run --coverage | Run the suite and report which lines ran. Coverage is not a quality score. |
npx vitest run --reporter=verbose | Print every test result, not only the failures. |
npx vitest run path/to/test.ts | Run one test file. Use it in PostToolUse for the file that was just edited. |
npx playwright test | Run all end-to-end browser tests. |
Plain vitest is a watch mode, not a single run. Use vitest run in the Stop hook, because a gate must finish and exit. In Jest, the same watch idea is jest --watch.
Testing Patterns
API Routes
- Test all CRUD operations
- Verify status codes (200, 201, 400, 401, 403)
- Include auth/authorization tests
- Test pagination and filtering
- Mock external services for speed
Auth Flows
- Login with valid/invalid credentials
- Protected routes reject unauthenticated requests
- Expired tokens force re-authentication
- Role-based access returns 403 for unauthorized roles
- Duplicate email signup is rejected
Payments (Stripe)
- Successful payment with test card
4242424242424242 - Generic decline:
4000000000000002 - Insufficient funds decline:
4000000000009995 - Verify webhook signature validation
- Test idempotency prevents double charges
- Bookly: a refund cannot exceed the deposit that was paid
Database Tests
- Use in-memory SQLite for fast tests
- Wrap each test in a transaction that rolls back
- Use fixtures/factories for pre-defined test data
- Every test starts with a clean state
- Never point tests at production data
UI Components
- Renders data correctly
- Shows loading state while fetching
- Shows error message on API failure
- Handles missing/null data gracefully
- Interactive elements trigger expected behavior
Secret Exposure
Write a test that scans your built output for patterns matching secret keys, API tokens, or database credentials.
"Write a test that verifies no env vars
containing SECRET, KEY, or PASSWORD
are included in the client-side bundle."
Mutation Score Instead of Coverage Targets
Measure What the Tests Catch
Coverage shows which lines ran. Mutation score shows which bugs the tests catch.
This guide gives no coverage percentage as a target. The published evidence points to mutation score, and the right number depends on the project. Use the steps below on the modules where a bug costs the most.
- Run npx vitest run --coverage to find code that no test touches.
- Run StrykerJS with its Vitest runner on the critical modules first: auth, payments and bookings.
- Send each surviving mutant to the agent as the next failing test. Ask for a test that fails on that mutant and passes on the original code.
- Run the suite again. Repeat until the surviving mutants on critical logic are removed or explained.
Install the Stryker Vitest runner with @stryker-mutator/vitest-runner. Stryker forces its coverage analysis to perTest and overrides several Vitest options, so read its Vitest notes before you add it to a hook.
100% Coverage Does Not Mean Zero Bugs
A test can run a function without one meaningful assertion. The line still counts as covered, and the bug still ships. Coverage shows what ran. It does not show what the tests would catch.
CI, Cross-Model Review and Flaky Tests
CI: The Same Gate on Every Push
Run the same npm test command the Stop hook runs
Set the test script in package.json to vitest run, so npm test runs once and exits. Create a file at .github/workflows/test.yml. CI is the place where a red suite stops a merge, because a hook can only retry once per stop.
# .github/workflows/test.yml
name: Tests
on: [push, pull_request]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: actions/setup-node@v7
with:
node-version: 22
- run: npm ci
- run: npm testLine-by-line breakdown:
What You See on GitHub
After setup, every commit shows a green checkmark (tests passed) or a red X (something broke). Mark the Tests check as a required check in the branch protection rules. A red X then blocks the merge, which makes CI the hard gate for a red suite.
Handling Flaky Tests
Tests that pass sometimes and fail at other times usually have one of three causes: (1) timing issues with async operations, (2) test order dependency, where tests share state, or (3) race conditions. Playwright auto-waits for all the relevant checks to pass before it performs an action. Vitest 5 adds a --repeats flag that runs a test many times, which exposes flakiness early. Copy the full error and ask the agent: "This test passes sometimes and fails sometimes. What might cause this?"
Playwright Healer Can Repair Tests Into Passing
Playwright 1.56 added three test agents: Planner, Generator and Healer. The Healer runs the suite and repairs failures. For TDD that is a risk, because a repaired test can pass for the wrong reason. Review every change the Healer makes to an assertion, and keep the rule the test states.
Cross-Model Review
Use a reviewer from a different model family
Addy Osmani's June 2026 post on agentic code review says that "a second model with different priors genuinely catches real bugs." He also warns that models from the same family can share correlated blind spots, so pick the reviewer from another family.
Phase 1: Create
Write the code and its tests with your primary model. The tests must be red before the code exists.
Phase 2: Review
Send the code and tests to a reviewer from a different model family, for example GPT-6.1 Sol in Codex. Ask for the weak spots only.
Phase 3: Fix
Every fix the reviewer proposes goes through a red test first. The original tests must still pass when the fix is done.
Anthropic's current lineup is Claude Fable 5.1, Opus 5.5, Sonnet 5.5 and Haiku 5.5. Codex's current models are GPT-6 Astra, GPT-6.1 Sol, GPT-6.1 Terra and GPT-6 Luna.
"These tests were written by another AI. Find all cases where: - Tests are sycophantic (confirm existing behavior rather than verifying correct behavior) - Tests check implementation details instead of observable behavior - Edge cases are missing (null inputs, empty arrays, boundary values, error conditions) - Assertions are too weak (toBeTruthy instead of specific value checks)"
Tools that run this review step on every pull request are compared on the workflow page, with their plan prices.
When to Skip Tests
Skip tests only for throwaway code, such as a one-time script or a spike. Bookly's booking, payment and refund paths are never throwaway.
Next step
The tests on this page check behavior. Bookly still needs checks that tests of behavior do not make on their own: a client who can read another client's bookings, an exposed key, and a payment webhook that nothing verifies. The next page, vibe coding security, covers those checks.
Common Questions
How do you do TDD with AI coding agents?
Write or approve one failing test, ask the agent for the minimum code that makes it pass, then refactor while the suite stays green. The rule that matters most is that the agent never edits the test. Enforce it with a hook, not a prompt, because Anthropic says CLAUDE.md rules are followed most of the time, and a test gate needs all of the time.
Can you trust unit tests written by AI?
Not without checking them. One study found a suite can reach 100% line coverage with a 4% mutation score, and another found LLM-written tests averaged a 40.21% mutation score on Python functions. Read each assertion, make it check a value a user would see, and run a mutation test on the critical paths such as Bookly's double-booking rule.
How do I stop AI from changing the test to make it pass?
A prompt works most of the time, so enforce the rule in a hook. In Claude Code, add a PreToolUse hook that exits with code 2 for any edit to a test file once the red phase is done. A read-only reviewer subagent can run with its tools restricted so it cannot write or edit. Bash can still write files, so cover shell commands with a hook or a deny rule too.
How do I make Claude Code run tests before it stops?
Read stop_hook_active from the hook's JSON input and exit 0 when it is true. That gives the agent one forced retry per stop, not an unbreakable gate. Claude Code overrides a Stop hook after it blocks eight times in a row with no tool call from Claude in between, and CLAUDE_CODE_STOP_HOOK_BLOCK_CAP raises that cap. Keep the hard gate in CI, as a required check that blocks the merge, and in a PreToolUse rule that blocks edits to test files.
Why does 100% test coverage still miss bugs in AI-written code?
Coverage counts lines that ran, not faults that the tests catch. One published analysis notes that some test suites reach 100% coverage with only a 4% mutation score. A benchmark of LLM-written unit tests on real Python functions averaged a 40.21% mutation score across the models tested. Run a mutation tool such as StrykerJS with its Vitest runner, then feed the surviving mutants back to the agent as new failing tests.
Does Vitest 5 break tests that an AI wrote?
It can. Vitest 5.0 shipped on September 3, 2026. It now fails a test that leaves an expect().resolves or .rejects assertion without await, and it sets clearMocks to true by default, so a test that counts calls made in a setup hook or an earlier test sees zero. Add the missing awaits and trigger the calls inside the asserting test.