It's 2:14am and checkout is failing. By 2:15 the investigation is running.
Join the waitlistAn alert fires. Polylane picks it up. Real or noise: answered in under a minute.
Elevated error rate on payments-api
- 02:14:07 Alert received
Datadog · payments-api error rate · occurrence #1
- 02:14:09 Triage started
Fetching the monitor query and metrics 30 min before, 5 min after the firing.
- 02:14:58 Triaged as an incident
Severity high. Error rate 41× the 30-minute baseline, clustered on POST /v1/charges; deploy f3a9c12 landed 18 minutes before the firing.
- 02:15:02 Investigation started
3 hypotheses · 3 passes each
- 02:31:44 Alert fired again (occurrence #2)
Folded into this issue after 16 quiet minutes; the running investigation was nudged.
- 03:06:12 Alert recovered
Provider recovery received. Issue resolved.
It digs until it can prove it. Hypotheses argued from both sides, verdicts that need evidence.
What I did
I queued an autofix that restores the Hyperdrive pool hd-prod in the coreplane/checkout-edge repo with:
max_connectionsrestored from 5 to 50 on the pool binding- a regression test that fails when checkout queues on the pool
wrangler.jsoncconfigured to deploy ascheckout-edge
This gives Boris a PR with deployable code. Once merged, running wrangler deploy will replace the degraded worker with the fixed version.
The root cause report has been updated with the occurrence #2 evidence: Root Cause: checkout-edge Latency Degradation (Recurring).
AI can make mistakes. Please double check cited sources.
The incident ends with a fix, not a follow-up meeting. Root cause found, fix written, ready for your review.
Cap retries on the checkout webhook worker #491
polylanemain from polylane/autofix/chat/k3x9f2-4e7d21a Retries on the checkout webhook worker were unbounded: a failing delivery re-queued itself forever and amplified load on payments-api. This caps delivery at 5 attempts with exponential backoff and dead-letters the payload after the last one.
What changed
worker/deliver.ts gains MAX_DELIVERY_ATTEMPTS = 5 and backoff between attempts; exhausted payloads land in checkout-webhooks-dlq instead of re-queueing.
Validation
npm test — 214 passed. A forced failing delivery stopped after 5 attempts and appeared in the dead-letter queue.
How Polylane works. It learns your system, watches it, investigates, and acts.
-
It learns your system first
The context graph maps every resource and dependency across your clouds, repos, and observability providers. Agents reason over real topology, not guesses.
-
Detection without thresholds
Built-in checks for every provider, plus checks generated from your own saved queries and dashboards. A statistical pass and an agent decide together, and an improvement never raises an issue.
-
Investigations that show receipts
Every claim links back to the query, log line, or change record behind it. A verdict without evidence falls back to inconclusive.
-
Writes are earned, never assumed
Accounts connect read-only. Rollbacks are off by default, rate-limited, and on the record. Code changes go through your normal review.
-
It gets sharper every week
Memories, daily notes, and monitoring queries re-confirmed against real data: July's investigation learns from June's.
It plugs into what you already run. Connect read-only and start.
Questions.
Does Polylane page me?
It's not a pager and doesn't try to be one. Polylane notifies over email, Slack, and the console, and only for critical and high severity issues it detected itself. The goal is fewer wake-ups: the issue arrives already investigated.
What happens when an issue is detected at night?
Triage confirms it's real, an investigation starts on its own, and hypotheses are tested in parallel against your actual telemetry. By morning the issue has a verdict, the evidence behind it, and suggested next steps; where autofix is enabled, the fix is already written and waiting for review.
Which alert sources does Polylane ingest?
Datadog, Honeycomb, Axiom, Better Stack, Sentry, CloudWatch, Vercel, Render, and Cloudflare, plus a generic webhook for everything else. Each source authenticates with a scoped telemetry token, and every firing becomes a triaged issue.
What if Polylane calls a real incident noise?
'No incident' is a verdict, not a deletion: the issue stays in the console with the reasoning attached, and you can start an investigation on anything with one click. Verdicts need evidence, and an investigation that couldn't reach data never confirms.
How many investigations will it run?
Automatic investigations are capped per rolling 24 hours: 10 on the Free plan, your own cap on paid plans. Manual starts never count against it. Pricing is public at polylane.com/pricing.