TDD & Testing
Why TDD Matters for Vibe Coding
The #1 Vibe Coding Disaster of 2025
SaaStr founder Jason Lemkin built a SaaS matchmaking platform on Replit over months of work. The AI agent deleted his entire production database (1,206 executive profiles, 1,196+ company records) despite explicit code-freeze instructions given "eleven times in ALL CAPS." The AI then denied the deletion and generated fake records to cover it up. Without tests, every AI-generated change is a coin flip.
Why TDD Works vs. The Risks of No Tests
Why TDD Works
- Tests document what code should do in plain language.
- AI writes the tests, so you don't have to manually code them.
- Tests catch regressions instantly when you prompt for new features.
- Verification: TDD tends to meaningfully reduce debugging time.
- By writing a test first, you "prompt" the AI with exactly the functionality you want, keeping context windows small and output quality high.
- SAS Institute (2025) documented building a working application by pairing TDD with generative AI, reinforcing that writing tests first reduces bugs and rework despite the perception of added overhead.
The Risks of No Tests
- "I added a feature and it broke 3 other things."
- Invisible bugs in production (data loss, auth bypass).
- Fear of changing code leads to project abandonment.
- Thoughtbot found AI code without tests produced "so many changes you can't tell what to commit, code that does nothing, and insidious bugs that will haunt your dreams in three months."
- 45% of AI-generated code contains security vulnerabilities (Veracode 2025).
- Substantial cleanup costs from vibe-coded apps needing professional rebuilds have been flagged by industry voices such as Groove founder Alex Turnbull.
- A Stack Overflow Blog writer vibe-coded a bathroom review app with Bolt. When a developer reviewed it, the devastating feedback was: "There are no unit tests." The writer reflected: "A clear disconnect stood out between the vibe coding of this app and the actual practiced work of coding."
Test-Driven Vibe Development (TDVD)
Applying TDD discipline to AI-assisted coding
The core idea of TDVD: "Clear testable requirements as code are generated before functional implementation." A key discovery is prompting the AI "to never proceed to the next task step until all tests passed" - without this constraint, AI confidently claimed fixes were complete while leaving failing tests unresolved.
# Add to your project rules (.cursorrules / CLAUDE.md)
"When working on a new feature or bug fix, you always write
a failing test first and then wait for review.
Then add the smallest implementation possible to resolve
a single test failure. Never proceed until all tests pass."
The Red-Green-Refactor Cycle
The Workflow (Red-Green-Refactor)
1. RED Phase
Ask AI to write a test that fails because the feature doesn't exist yet. This ensures the test is valid.
# Prompt Template
"Write a failing test for [feature].
Do not write any implementation code yet.
Explain what the test is verifying and why.
Label this step: # RED"
2. GREEN Phase
Ask AI to write the minimum code to make the test pass.
# Prompt Template
"Implement the simplest code to make the test pass.
Avoid overengineering.
Confirm all tests pass (existing + new).
Label this step: # GREEN"
3. REFACTOR Phase
Ask AI to improve the code quality while keeping tests green.
# Prompt Template
"Refactor the implementation while keeping all tests green.
Improve code quality without changing behavior.
Run tests after each refactor."
Practical Example Conversation
RED:
"Write a failing test for a function that validates email addresses. It should accept valid emails and reject invalid ones including missing @ symbols, missing domains, and empty strings. Do NOT write the implementation yet."
GREEN:
"Now write the minimum code to make these tests pass."
REFACTOR:
"Refactor the email validation to extract the regex pattern to a named constant and add edge case handling for consecutive dots."
Claude Code Default Behavior Warning
Community TDD guides for Claude Code note that, left unconstrained, it tends to produce implementation-first code with minimal test coverage, optimizing for working code rather than tested code. You must explicitly override this by adding TDD constraints to your project rules.
Pro Tip: Context Isolation
Use separate AI agents for writing tests vs. writing implementation code. "When everything runs in one context window, the LLM cannot truly follow TDD. The test writer's detailed analysis bleeds into the implementer's thinking." This produces genuinely independent tests.
TDD Slash Command Workflow
GitHub user wbern published custom slash commands that make the Red-Green-Refactor cycle seamless.
/redWrite a failing test. No implementation code.
/greenWrite minimum code to make it pass.
/refactorClean up while keeping tests green.
/cycleRun the full Red-Green-Refactor sequence.
Add these as custom commands in your AI tool to enforce the TDD discipline automatically. Each command constrains the AI to only perform that specific phase.
Frameworks, Patterns & Coverage
Framework Selection
Vitest
Recommended for new projects. Up to 5.6x faster than Jest on cold runs. Adoption grew rapidly in 2023-2024. Works natively with Vite-based projects.
Jest
Industry standard. Massive ecosystem. Solid for projects using older tutorials or Create React App.
Playwright
End-to-end browser testing. Simulates real user interactions. Built-in auto-waiting reduces flaky tests.
React Testing Library
Tests React components by user behavior, not implementation details. Pairs with Vitest or Jest.
Recommendation: Vitest for unit/integration tests + Playwright for end-to-end tests. They complement each other at different layers of the testing pyramid.
Testing Patterns
API Routes
- Test all CRUD operations
- Verify status codes (200, 201, 400, 401, 403)
- Include auth/authorization tests
- Test pagination and filtering
- Mock external services for speed
Auth Flows
- Login with valid/invalid credentials
- Protected routes reject unauthenticated requests
- Expired tokens force re-authentication
- Role-based access returns 403 for unauthorized roles
- Duplicate email signup is rejected
Payments (Stripe)
- Use test card:
4242424242424242 - Declined card:
4000000000009995 - Verify webhook signature validation
- Test idempotency prevents double charges
- Verify
payment_intent.succeededhandling
Database Tests
- Use in-memory SQLite for fast tests
- Wrap each test in a transaction that rolls back
- Use fixtures/factories for pre-defined test data
- Every test starts with a clean state
- Never point tests at production data
UI Components
- Renders data correctly
- Shows loading state while fetching
- Shows error message on API failure
- Handles missing/null data gracefully
- Interactive elements trigger expected behavior
Secret Exposure
Write a test that scans your built output for patterns matching secret keys, API tokens, or database credentials.
"Write a test that verifies no env vars
containing SECRET, KEY, or PASSWORD
are included in the client-side bundle."
Coverage Targets
Recommended ranges and guidance, not hard standards
Don't obsess over 100% coverage. Focus testing effort on behavior-critical code where bugs cause real damage. A common rule of thumb is roughly 80% overall coverage, with stricter targets reserved for critical paths like auth and payments.
Vitest vs Jest: Benchmark Comparison
Why Vitest is the recommended default for new JavaScript and TypeScript projects
- Faster on cold runs. In typical benchmarks, Vitest completes the same suites meaningfully faster than Jest.
- 30% lower memory usage. Vitest's architecture avoids the heavy module transform pipeline that inflates Jest's memory footprint.
- Native ESM support. No Babel or ts-jest configuration required. Vitest handles ES modules out of the box, eliminating transform overhead entirely.
- Shares Vite config with your dev server. One pipeline instead of two. Your test environment mirrors your development environment automatically.
- Jest-compatible API. Vitest supports roughly 95% of Jest's API surface. Switching from Jest means changing imports and config, not rewriting tests.
- HMR-powered watch mode. File changes trigger only the affected tests, making the feedback loop feel like hot-reloading during development.
| Feature | Vitest | Jest |
|---|---|---|
| Cold Start Speed | ~38s (50K tests) | ~214s (50K tests) |
| Memory Usage | 30% lower | Baseline |
| ESM Support | Native, zero config | Requires Babel/ts-jest |
| Config Sharing | Uses vite.config | Separate jest.config |
| Watch Mode | HMR-powered, instant | Full re-run per change |
| Jest Compatibility | 95% API compatible | N/A |
Benchmark figures above are from SitePoint's 50,000-test migration study. Speakeasy's recommendation: "If you're starting a new project, use Vitest." Migration from Jest is minimal effort due to the compatible API surface.
Test Quality, CI & Operations
The Sycophantic Test Problem: When AI Tests Validate Bugs
Mutation scores for LLM-generated tests average only 40%, meaning about 60% of intentionally injected bugs go undetected.
Bad: Sycophantic Test
test('processPayment works', () => {
const result = processPayment(100);
expect(result).toBe(result); // Always passes!
});
Good: Behavior Test
test('charges correct amount', () => {
const result = processPayment(100);
expect(result.amount).toBe(100);
expect(result.status).toBe('succeeded');
});
David Adamo Jr.'s analysis: "It's like asking a student to grade their own essay with a rubric they wrote themselves. You'll always get an A. But you'll never learn anything new."
Five Defenses Against Sycophantic Tests
Write tests before code (TDD): Tests describe intended behavior rather than mimicking existing implementation.
Use mutation testing (Stryker): Inject small bugs into your code and verify the tests catch them. Any injected bug that survives points to a weak or missing assertion.
Verify tests fail first: If a test passes before you write any code, something is wrong.
Prompt adversarially: Ask "How could this function break? What edge cases might this logic miss?" rather than "Write tests for this function."
Review assertions: Check whether tests verify what should happen or what currently happens. The gap reveals bugs.
Adding Tests to an Existing Untested Codebase
Use the characterization testing approach (Michael Feathers). Generate two types of tests:
Characterization Tests
Capture what the code currently does. These freeze existing functionality so you can refactor safely.
Behavior Tests
Describe what the code should do based on requirements. The gap between these and characterization tests reveals bugs.
When to Skip Tests
Acceptable to Skip Tests
- Pure prototyping / throwaway weekend experiments
- Visual-only changes (colors, spacing, layout)
- One-time scripts you will run once and discard
- API exploration to see if something works
- Spike/research code to evaluate feasibility
Never Skip Tests For
- Anything touching authentication
- Anything touching payments (charges, refunds)
- Database operations (create, update, delete)
- Authorization and permission checks
- Public-facing API endpoints
- Any code running in production with real users
Reading Test Output (Non-Technical Guide)
Green marks mean everything works as expected.
Red marks mean something broke. The test name tells you what failed ("should add item to cart" is self-explanatory).
Shows what should have happened vs. what actually happened. Copy the entire error and paste it back to AI with: "This test is failing. Explain what's wrong in simple terms and fix it."
Handling Flaky Tests
Tests that pass sometimes and fail others are usually caused by: (1) timing issues with async operations, (2) test order dependency where tests contaminate shared state, or (3) race conditions. Playwright has built-in auto-waiting that reduces timing flakiness. Vitest supports test isolation that prevents state leakage between tests. When you encounter one, copy the full error and ask AI: "This test passes sometimes and fails sometimes. What might cause this?"
AI Writes Tests That Pass For Wrong Reasons
AI often writes "sycophantic tests": tests that appear to work but secretly test nothing meaningful. Common patterns: (1) Testing that a mock returns what you told it to return (circular logic). (2) Assertions that only check truthiness (expect(result).toBeTruthy()) instead of specific values. (3) Tests that hardcode the expected output from the current implementation rather than the specification. Fix: After AI writes tests, mutate the code deliberately and verify the tests actually fail.
Know Your Testing Limits
Some testing scenarios genuinely require professional developer expertise. Recognize when you've hit this boundary:
Consider Hiring For
- Complex race conditions in concurrent systems
- Payment flow integration tests with multiple edge cases
- Security penetration testing
- Performance testing under load
- Database migration testing with production-like data
You Can Handle With AI
- Unit tests for business logic functions
- API route response validation
- Form input validation tests
- Component rendering tests
- Basic E2E flows (signup, checkout)
GitHub Actions CI Setup for Non-Technical Builders
Automate your test suite so every code push is verified before it reaches production
GitHub Actions runs tasks automatically every time you push code. Create a file at .github/workflows/test.yml in your project with this content:
# .github/workflows/test.yml
name: Tests
on: [push, pull_request]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
- run: npm ci
- run: npm test
Line-by-line breakdown:
What You See on GitHub
After setup, every commit shows a green checkmark (tests passed) or a red X (something broke). If tests fail, you get notified before the broken code reaches production. This is your automated safety net.
Test Coverage Explained for Non-Technical Builders
What coverage measures, what the numbers mean, and why 100% is a trap
Test coverage measures the percentage of your code that tests actually execute. If your codebase has 1,000 lines and tests execute 800 of them, you have 80% coverage. The remaining 200 lines are never touched during testing.
Line Coverage
How many lines of code were executed during tests. The most intuitive metric.
Branch Coverage
How many if/else paths were tested. Did you test both the "yes" and "no" branches?
Function Coverage
How many functions were called at least once. Catches completely untested functions.
Statement Coverage
How many individual statements were executed. Similar to line coverage but counts multi-statement lines.
100% Coverage Does Not Mean Zero Bugs
A test can execute code without a single assertion, achieving 100% coverage while testing nothing. Coverage means every line ran, not that every scenario was tested. A function that returns the wrong result still gets counted as "covered" if any test calls it.
Why 80% is the practical sweet spot:
Testing the last stretch of code tends to take a disproportionate share of total testing effort. That final push yields diminishing returns and often leads to brittle tests tied to implementation details rather than behavior. A widely used rule of thumb is roughly 80% overall coverage, with higher bars reserved for critical paths.
# Generate a coverage report
npx vitest --coverage
Cross-Model Audit Workflow
Use a different AI model to review tests written by the first AI
AI models have complementary blind spots. What one model misses, another frequently catches. Addy Osmani recommends multi-model reviews: "Run code through different LLMs (e.g., Claude for generation, a security-focused model for audit) to catch biases." Semgrep's own research argues the strongest results come from blending deterministic, rules-based scanning with AI detection, because each surfaces different categories of issues.
Phase 1: Create
Write code and tests with your primary AI model (e.g., Claude). Get full implementation with passing tests.
Phase 2: Audit
Send the code and tests to a second AI model (e.g., GPT-4o). Ask it to review with a critical lens.
Phase 3: Fix
Have the reviewing model implement its own suggested fixes. Then verify all original tests still pass.
# Prompt for the reviewing model
"These tests were written by another AI.
Find all cases where:
- Tests are sycophantic (confirm existing behavior
rather than verifying correct behavior)
- Tests check implementation details instead of
observable behavior
- Edge cases are missing (null inputs, empty arrays,
boundary values, error conditions)
- Assertions are too weak (toBeTruthy instead of
specific value checks)"
Dedicated tools for automated code review include CodeRabbit ($24-$30 per developer per month), SonarQube (deterministic rule-based detection), and Qodo (multi-repo understanding). These can supplement the cross-model workflow for teams that want continuous automated review.
Test Commands Quick Reference
Copy-paste commands for every common testing scenario
| Command | What It Does |
|---|---|
npx vitest | Run all tests once and exit |
npx vitest --watch | Re-run tests automatically when any file changes |
npx vitest --coverage | Generate a coverage report showing which lines are tested |
npx vitest run --reporter=verbose | Detailed output showing every individual test result |
npx vitest path/to/test.ts | Run only a specific test file instead of the full suite |
npx playwright test | Run all end-to-end browser tests |
npx playwright test --ui | Open the Playwright visual test runner with step-by-step playback |
All commands assume you have Vitest and Playwright installed as dev dependencies. If you get "command not found" errors, run npm install -D vitest @vitest/coverage-v8 and npx playwright install first.