We asked Claude Code to check a webhook handler. Every delivery got a 200. Four cases were broken.
We gave Claude Code a small Go service and one sentence:
Check the Acme Pay webhook handling of this service for edge cases.
No hints, no list of bugs. Two minutes later it came back with a table of
eight cases, four of them failing. Every webhook the service had accepted
was answered with 200 OK. The bugs only showed up in what the service
did afterwards.

This post is about how it got there, what it found, and why checking the status codes would have found nothing.
The service
examples/agent-demo
is a subscription service of about 260 lines of Go. A customer subscribes, the
service creates a payment at Acme Pay, and Acme Pay reports the outcome
with signed webhooks: payment.pending, then payment.completed, and in
the bad case a payment.chargeback 30 days later. On completed the
customer gets access and credits; on a chargeback they lose both.
The service verifies the Acme-Signature header correctly, keeps accounts
in a JSON file, and answers 503 when it cannot save one, so that Acme Pay
retries. Its unit tests pass: the normal flow with each event sent once,
in order; a restart; a failed save; an unsigned webhook.
We planted two bugs and wrote nothing about them in the code or the README:
- The
completedhandler adds the amount without remembering the event id, so a redelivered event credits twice. - The
pendinghandler sets the status topendingunconditionally, so apendingthat arrives aftercompletedtakes access away from a customer who paid.
Acme Pay does not exist. Its API and webhooks are a
TwinStub scenario in one YAML
file: create a payment, queue pending after 30 seconds and completed
after two minutes, queue a chargeback 30 days after that.
What the agent did
Claude Code had the TwinStub plugin
installed. The plugin adds 13 MCP tools and a skill called
webhook-edge-cases. The agent loaded the skill on its own, because the
request was about webhooks.
The skill asks for one thing before anything runs: a table of “events in
this order, expected end state of our app”. The agent wrote eight rows:
the normal flow, completed delivered twice, completed before
pending, a bad and a missing signature, a chargeback 30 days later, the
chargeback delivered twice, a failed save followed by a retry, and an old
payment’s chargeback arriving after the customer had subscribed again.
Then it started the service and went through the table with the TwinStub tools:
twinstub_start_sessionto begin a payment, then a realPOST /subscribeagainst the service;twinstub_deliverto send the queued webhooks in a chosen order, for examplecompletedfirst;twinstub_replay_deliverywith moderedeliverto send the same event again under a new delivery id, and modesbad_signatureandno_signature;twinstub_advance_timeto move the session’s clock 30 days forward, so the chargeback arrived in one call;curl localhost:3000/accounts/u1after every step, to read what the service actually did.
For the retry row it made the data file read-only, delivered completed
(the service answered 503 and TwinStub queued a retry), restored the
file, and moved the clock two minutes forward so the retry fired.
What it found
| # | Case | Expected | Actual | |
|---|---|---|---|---|
| 1 | pending, then completed | active, 900 credits | active, 900 credits | pass |
| 2 | completed delivered twice | 900 credits | 1800 credits | fail |
| 3 | completed before pending | active | pending, no access | fail |
| 4 | bad or missing signature | rejected | 400, account unchanged | pass |
| 5 | chargeback 30 days later | revoked | revoked | pass |
| 6 | chargeback delivered twice | 0 credits | subtracted twice | fail |
| 7 | 503 on save, then a retry | applied once | applied once, but see below | pass / fail |
| 8 | old payment’s chargeback after resubscribing | new plan unaffected | paid plan revoked | fail |
Both planted bugs were there: rows 2 and 6 are the missing event-id check,
row 3 is the late pending. Row 8 we had not planted. The handler finds
the account by the payment’s reference and never compares the event’s
payment_id with the payment it is about, so a late event from an old
payment changes the current subscription. That is a real bug in the demo,
and the agent was right to report it.
Row 7 is the one we like most. The retry itself worked: the event was
applied exactly once. But the retry delays put the pending retry after
the completed retry, so bug 2 happened on its own, with nothing staged.
In the agent’s words:
retry backoff put the pending retry after completed, so the row 3 bug happened on its own
That is exactly how out-of-order delivery shows up in production: not because a provider is careless, but because retries after an outage do not keep the original order.
The agent also said what it had not tested and why, and proposed fixes:
store processed event ids in the same save as the account change, ignore
events whose payment_id is not the current payment, and only let the
status move forward. It changed no code, because we had only asked it to
check.
Why the status codes said nothing
Every delivery that passed the signature check got a 200. A test that
only asserts “the webhook was accepted” passes on all four failing rows. The
difference between a correct handler and this one is entirely in the
state it leaves behind: credits, status, what the customer can do.
That is why the skill tells the agent to write the expected end state first and to read the app’s state after each step. It is also why the expected state has to be written down before running anything: a snapshot of what the handler produced the first time would have recorded 1800 credits as correct.
What the runs taught us about TwinStub
We ran this three times, each time on a fresh copy, and each run found both planted bugs, in 3 minutes 41 seconds, 1 minute 50 seconds and 2 minutes 12 seconds. The first run also found three problems in TwinStub itself, which we fixed before the release:
- A plugin installed for every project started an engine in every Claude Code session, including projects that never use TwinStub. The engine is now started on the first tool call that needs it.
- When the app called the provider before a session existed, TwinStub answered “no rule matched”, although a rule did match. The 404 now says that no session exists and how to start one.
- The engine the plugin starts listened on all interfaces. It now listens on this machine only.
Try it
You need TwinStub 1.1.2 or later on your PATH (download), then in Claude Code:
/plugin marketplace add twinstub/claude-plugin
/plugin install twinstub@twinstub
Copy examples/agent-demo somewhere outside the repository, so the agent
cannot read about the bugs, open Claude Code in it and ask the same
question. Or point it at your own webhook handler: the skill works with
any provider you can describe in a TwinStub scenario.
One honest limit. TwinStub moves its own clock, not your app’s. If your handler has time rules of its own, such as a refund window, it needs an injectable clock to be tested the same way, and the skill tells the agent to say so instead of claiming it was tested.