Your first page shouldn’t be your first incident.
oncallsim puts you on call for a broken production system, with the incident already on the graphs. Find the cause, fix the code, and prove the fix under the traffic that paged you.
OrdersApiHighLatency
p95 latency of orders-api above 500 ms for 1 minute
histogram_quantile(0.95, …) > 0.5 · for 1m
How it works
Three stages, one session
-
1Diagnose
Read the metrics, logs and traces in Grafana. Name the service that failed and how it failed.
passes when the culprit and failure class are right
-
2Fix
Clone the repo, patch it in your own editor, push. The sandbox rebuilds and runs the regression suite.
passes when build OK · tests green
-
3Replay
We replay the original incident traffic against your fix. The alert that paged you has to stay quiet.
passes when the alert stays silent for the whole replay window
Every incident is timed against a par. Clear it for gold, silver or bronze.
Who it’s for
- First rotation
- You’re joining an on-call rotation and have never debugged production at 3 a.m.
- Interview prep
- You want real reps before an SRE or platform interview, not flashcards.
- After the fact
- You only ever read postmortems. This time you’re the one holding the pager.
Questions before your first round
What incidents can I work?
The first three:
- Checkout is timing outAn N+1 query exhausting the database connection pool.
Go - Worker keeps restartingA memory leak ending in OOM restarts.
Python - Gateway is throwing 5xxA retry storm cascading across services.
Go
Is this a quiz?
No. You get a running system, its Grafana, and its repository. You pick the cause from what the telemetry shows, write the fix yourself, and the grade is whether production holds.
Is the incident real?
It was recorded from a real run of the broken system and is replayed into your own Grafana when your session starts. The replay at the end is live traffic against your code.
What do I need?
A browser, git, and the editor you already use. Nothing to install from us.
Which languages?
Go and Python to start.
How long does a session take?
A session runs for up to two hours. Each incident has a par time, and beating it is how you earn gold.
What does it cost?
Pricing will be announced before public launch; the waitlist hears first.
When can I play?
Access opens in waves. The form is the queue.
What happens to my email?
It’s used to send your invite and nothing else.