Debugging an AI-generated codebase you have never read

The alert fires at 09:40. A checkout webhook is writing duplicate rows, roughly one in fifty, and you own the service. You wrote none of it. The history is eleven commits titled "wip" and one titled "fix", and the file you are staring at is seven hundred lines you have never opened.

Debugging code you did not write is an old skill and most of it transfers. The part that does not transfer is that generated codebases fail in a particular set of ways, and knowing those ways is worth several hours.

Make it reproducible before you touch anything

Nothing else works until the bug happens on demand. Not "usually", not "when I click around". A command you can run that fails.

The temptation is to skip this because the fix looks obvious from the stack trace. Resist it for one specific reason: in a codebase you do not know, you cannot otherwise tell whether your change fixed it, did nothing, or moved the symptom somewhere you have not looked. A failing command is the only instrument you have.

# Reproduce narrowly and repeatedly before changing anything.
npx vitest run src/webhooks/checkout.test.ts -t "duplicate" --reporter=verbose

# If it looks flaky, prove it is flaky rather than assuming.
for i in $(seq 1 20); do
  npx vitest run src/webhooks/checkout.test.ts -t "duplicate" \
    --reporter=dot >/dev/null 2>&1 && echo pass || echo FAIL
done | sort | uniq -c

Twenty runs takes two minutes and tells you whether you are chasing a logic bug or a race. That distinction determines everything you do next, and getting it wrong by assumption costs an afternoon.

Map before you fix

Now use the agent for what it is genuinely excellent at: reading a lot of unfamiliar code quickly and telling you where things connect.

Ask for the call path, not the fix. Something close to: "Trace the path from the POST /webhooks/checkout route handler to the database insert. List every file and function in order with line numbers, and say where the request could be processed twice. Do not edit anything."

Then verify. Open two or three of the files it named and check the functions exist, that they call what it says they call, and that the line numbers are roughly right. Four minutes. The map is usually good and it is occasionally a confident work of fiction assembled from plausible file names, and you have no other way to tell one from the other. Check a couple of road signs before you trust the map.

Bisect by behaviour

With a repro and a rough map, stop reasoning and start bisecting. Binary search along the path: pick the midpoint, assert what you believe is true there, run the repro.

// Halfway down the path. Not a fix — an instrument.
console.error("[bisect] handler entry", {
  eventId: event.id,
  idempotencyKey: req.headers["idempotency-key"] ?? null,
  alreadySeen: seenEvents.has(event.id),
});

Three or four rounds of that localises almost anything. If the history is usable, git bisect does the same job across time rather than across the call path, and it works fine on generated history as long as the commits build.

git bisect start
git bisect bad HEAD
git bisect good v1.4.0
git bisect run npx vitest run src/webhooks/checkout.test.ts -t "duplicate"

The catch with agent-heavy history is commit size. When each commit is a nine-hundred-line feature drop, bisect narrows you to which afternoon broke it. Better than nothing, considerably worse than a line.

Names are decoration until proven otherwise

In generated code a name is a hypothesis about what a function does, formed at the moment it was named and never revisited afterwards. validateUser may check one field and return true. retryWithBackoff may retry three times with no backoff. sanitiseInput may trim whitespace.

Read the body. Every time, for every function on the critical path. It feels slow and it is the highest-yield habit available here, because a wrong belief formed from a name will survive three hours of investigation without ever being tested.

Check you are editing the file that actually runs

Generated codebases accumulate near-duplicates: two modules doing almost the same job, written in different sessions, one of which nothing imports. Fixing the unused one is a genuinely common way to lose an afternoon. Your change is correct, your test still fails, and nothing about the situation hints at why. The duplicate code problem covers how the copies accumulate in the first place.

Find the real importers before you edit.

# Who actually imports this module?
rg -n "from ['\"].*checkout-handler" --glob '!node_modules' --glob '!dist'

# Are there siblings doing the same job under another name?
rg -l "idempotency" --glob '!node_modules' --glob '!dist' | sort

If you are still unsure whether a file is live, make it throw and run the repro. Nothing crashes, and you have found your real problem.

The seams where things go quiet

Generated code is defensive in a way that hides failure. It catches, it defaults, it falls back rather than stopping. Each of those is a place where the real error was discarded before it could reach you.

// Four silent failures in nine lines.
export async function getRate(currency: string): Promise<number> {
  try {
    const base = process.env.RATES_URL ?? "http://localhost:3000";
    const res = await fetch(`${base}/rates`);
    const data = await res.json();       // no res.ok check
    return data.rates?.[currency] ?? 1;  // missing rate becomes 1
  } catch {
    return 1;                            // network failure becomes 1
  }
}

Every path returns 1, and 1 is a perfectly plausible exchange rate. Grep for the shapes:

rg -n "catch\s*(\(\s*\w*\s*\))?\s*\{\s*\}" --glob '*.ts'    # empty catch
rg -n "\?\?\s*(0|1|\[\]|\{\}|''|\"\")" --glob '*.ts'         # defaulting away absence
rg -n "process\.env\.\w+\s*(\?\?|\|\|)" --glob '*.ts'        # env with silent default
rg -n "mockResolvedValue|mockReturnValue" --glob '!**/*.test.ts'

Run that last one on any generated service. Mocks written during scaffolding and never replaced with real calls do not announce themselves. They return the same cheerful fixture forever.

Working with the agent once you are inside it

Debugging is where an agent is most useful and most dangerous, because it is delighted to try fixes. Three rules keep it on the useful side.

Paste the stack trace verbatim, all of it, not your summary. Your summary is already an interpretation, and the interpretation is what you are stuck inside.

Ask for ranked hypotheses with distinguishing evidence. "Give me three hypotheses for why this insert runs twice, ordered by likelihood. For each, state the single observation that would confirm or eliminate it. Do not edit any files." That reframes the turn from producing a diff to designing an experiment.

Require proof before edits. A hypothesis earns one log line, you run the repro, the log supports it or it does not. Only then does anything change. What you are preventing is the sequence where it tries fix one, fails, stacks fix two on top, fails, and four rounds later you are debugging four half-fixes layered on each other instead of the original bug — see The retry loop: spotting it and breaking it for how that starts and how to get out.

All of it reduces to one rule. In a codebase you have not read, no belief counts until you have observed it: not the function name, not the agent's map, not the comment above the line, not your own reasonable inference about what the code probably does. A failing command, a log line, a crash you caused on purpose. Everything else is a guess with good handwriting.