We asked Claude Code to check a webhook handler. Every delivery got a 200. Four cases were broken.

November 9, 2026 · 6 min read

We gave Claude Code a small Go service and one sentence:

Check the Acme Pay webhook handling of this service for edge cases.

No hints, no list of bugs. Two minutes later it came back with a table of eight cases, four of them failing. Every webhook the service had accepted was answered with 200 OK. The bugs only showed up in what the service did afterwards.

Claude Code with the TwinStub plugin checks a webhook handler and finds four failing cases

This post is about how it got there, what it found, and why checking the status codes would have found nothing.

The service

examples/agent-demo is a subscription service of about 260 lines of Go. A customer subscribes, the service creates a payment at Acme Pay, and Acme Pay reports the outcome with signed webhooks: payment.pending, then payment.completed, and in the bad case a payment.chargeback 30 days later. On completed the customer gets access and credits; on a chargeback they lose both.

The service verifies the Acme-Signature header correctly, keeps accounts in a JSON file, and answers 503 when it cannot save one, so that Acme Pay retries. Its unit tests pass: the normal flow with each event sent once, in order; a restart; a failed save; an unsigned webhook.

We planted two bugs and wrote nothing about them in the code or the README:

  1. The completed handler adds the amount without remembering the event id, so a redelivered event credits twice.
  2. The pending handler sets the status to pending unconditionally, so a pending that arrives after completed takes access away from a customer who paid.

Acme Pay does not exist. Its API and webhooks are a TwinStub scenario in one YAML file: create a payment, queue pending after 30 seconds and completed after two minutes, queue a chargeback 30 days after that.

What the agent did

Claude Code had the TwinStub plugin installed. The plugin adds 13 MCP tools and a skill called webhook-edge-cases. The agent loaded the skill on its own, because the request was about webhooks.

The skill asks for one thing before anything runs: a table of “events in this order, expected end state of our app”. The agent wrote eight rows: the normal flow, completed delivered twice, completed before pending, a bad and a missing signature, a chargeback 30 days later, the chargeback delivered twice, a failed save followed by a retry, and an old payment’s chargeback arriving after the customer had subscribed again.

Then it started the service and went through the table with the TwinStub tools:

  • twinstub_start_session to begin a payment, then a real POST /subscribe against the service;
  • twinstub_deliver to send the queued webhooks in a chosen order, for example completed first;
  • twinstub_replay_delivery with mode redeliver to send the same event again under a new delivery id, and modes bad_signature and no_signature;
  • twinstub_advance_time to move the session’s clock 30 days forward, so the chargeback arrived in one call;
  • curl localhost:3000/accounts/u1 after every step, to read what the service actually did.

For the retry row it made the data file read-only, delivered completed (the service answered 503 and TwinStub queued a retry), restored the file, and moved the clock two minutes forward so the retry fired.

What it found

#CaseExpectedActual
1pending, then completedactive, 900 creditsactive, 900 creditspass
2completed delivered twice900 credits1800 creditsfail
3completed before pendingactivepending, no accessfail
4bad or missing signaturerejected400, account unchangedpass
5chargeback 30 days laterrevokedrevokedpass
6chargeback delivered twice0 creditssubtracted twicefail
7503 on save, then a retryapplied onceapplied once, but see belowpass / fail
8old payment’s chargeback after resubscribingnew plan unaffectedpaid plan revokedfail

Both planted bugs were there: rows 2 and 6 are the missing event-id check, row 3 is the late pending. Row 8 we had not planted. The handler finds the account by the payment’s reference and never compares the event’s payment_id with the payment it is about, so a late event from an old payment changes the current subscription. That is a real bug in the demo, and the agent was right to report it.

Row 7 is the one we like most. The retry itself worked: the event was applied exactly once. But the retry delays put the pending retry after the completed retry, so bug 2 happened on its own, with nothing staged. In the agent’s words:

retry backoff put the pending retry after completed, so the row 3 bug happened on its own

That is exactly how out-of-order delivery shows up in production: not because a provider is careless, but because retries after an outage do not keep the original order.

The agent also said what it had not tested and why, and proposed fixes: store processed event ids in the same save as the account change, ignore events whose payment_id is not the current payment, and only let the status move forward. It changed no code, because we had only asked it to check.

Why the status codes said nothing

Every delivery that passed the signature check got a 200. A test that only asserts “the webhook was accepted” passes on all four failing rows. The difference between a correct handler and this one is entirely in the state it leaves behind: credits, status, what the customer can do.

That is why the skill tells the agent to write the expected end state first and to read the app’s state after each step. It is also why the expected state has to be written down before running anything: a snapshot of what the handler produced the first time would have recorded 1800 credits as correct.

What the runs taught us about TwinStub

We ran this three times, each time on a fresh copy, and each run found both planted bugs, in 3 minutes 41 seconds, 1 minute 50 seconds and 2 minutes 12 seconds. The first run also found three problems in TwinStub itself, which we fixed before the release:

  • A plugin installed for every project started an engine in every Claude Code session, including projects that never use TwinStub. The engine is now started on the first tool call that needs it.
  • When the app called the provider before a session existed, TwinStub answered “no rule matched”, although a rule did match. The 404 now says that no session exists and how to start one.
  • The engine the plugin starts listened on all interfaces. It now listens on this machine only.

Try it

You need TwinStub 1.1.2 or later on your PATH (download), then in Claude Code:

/plugin marketplace add twinstub/claude-plugin
/plugin install twinstub@twinstub

Copy examples/agent-demo somewhere outside the repository, so the agent cannot read about the bugs, open Claude Code in it and ask the same question. Or point it at your own webhook handler: the skill works with any provider you can describe in a TwinStub scenario.

One honest limit. TwinStub moves its own clock, not your app’s. If your handler has time rules of its own, such as a refund window, it needs an injectable clock to be tested the same way, and the skill tells the agent to say so instead of claiming it was tested.