A bug hunt on Osprey: 4 bugs proven, 2 shipped
On 15 November 2026, taskwing hunt found 4 bugs in Osprey, an open source transaction monitoring engine in Go, proved each with a test that failed on the code, and fixed all 4 in parallel sandboxes in 8 minutes 58 seconds, passing 5 of 5 exam checks. 2 of the 4 had already been fixed on main, because the hunt ran on an older checkout. The other 2 shipped in Osprey pull request #120 after review.
The setup
Osprey is an open source transaction monitoring engine: CEL rules, FATF typologies, one Go binary. It has a real test suite and a public history, which made it a fair target. We ran the TaskWing proof of concept against it on 15 November 2026, with Claude Code as the agent and boat for the sandboxes.
The receipt
| Measure | Result |
|---|---|
| Bugs proven by a failing test | 4 |
| Fixer agents, one sandbox each | 4 |
| Time, hunt to fixed branch | 8 min 58 s |
| Exam on the joined work | 5 of 5 checks passed |
| GitHub credentials in any sandbox | None |
| Already fixed on main | 2 (in Osprey #115 and #118) |
| Shipped | 2, in Osprey #120, merged 15 November 2026 |
The two bugs that shipped
1. The bus kept delivering after Unsubscribe
In the in-memory event bus, Unsubscribe only cancelled a context. The subscription stayed registered, so Publish still queued messages for it and the handler could still run. The failing test did 300 rounds of subscribe, unsubscribe, publish. On main, the handler ran 289 times after it had unsubscribed, and all 300 subscriptions stayed registered. After the fix: 0 late calls, 0 left registered.
2. Times stored with the caller's offset
The repository stored transaction times with whatever offset the caller passed, and SQLite compares them as text against UTC bounds. A transaction at 12:00 UTC, saved as 14:00+02:00, fell outside the window 11:59 to 12:01 UTC, and the received-time window failed with a scan error. After the fix, times are stored in UTC and both windows find it. The HTTP API and the worker already converted to UTC, so only direct callers of the repository were affected.
The two that didn't ship
A sanctions name-variant miss and a 409 on an evaluate retry were real bugs on the code the hunt saw. But that checkout was behind main, and both had already been fixed in Osprey #115 and #118. The hunt did its job; our setup did not. TaskWing will hunt on the latest main and skip bugs that are already fixed or in an open pull request.
What we took from it
- "No failing test, no bug" works. We reviewed 4 proven bugs, not a list of maybes. Nothing in the report was an opinion.
- Fixers should not see the test. Each fixer got the bug in plain words. The exam then ran the hidden tests on the joined work, plus the existing suite.
- A person still reviews. We read every fix before it shipped. Fixes that touch scoring, a public API or a migration deserve a maintainer's eye, and flagging them automatically is planned. TaskWing stops at a draft pull request for exactly this reason.
Questions
What is Osprey?
An open source transaction monitoring engine for financial crime detection, written in Go and licensed Apache-2.0, maintained by opensource.finance.
Were the bugs real?
Yes. Each was proven by a test that failed on the code at the time. Two were already fixed on main by the time we looked; the other two are fixed in pull request #120, each with its test.
Did a person review the fixes?
Yes. The pull request was reviewed and merged by hand. TaskWing never merges; a run ends at a draft pull request.
Why did the hunt find bugs that were already fixed?
It ran on a local checkout that was behind main. Hunting on the latest main, and skipping bugs already fixed or in open pull requests, is planned.
Try the proof of concept
TaskWing is open source (AGPL-3.0) and early. Today it runs Claude Code agents in boat sandboxes, or in local folders to try the flow.
go install github.com/josephgoksu/TaskWing/cmd/taskwing@main
taskwing run "add a /health endpoint with a test" --provider local --no-pr