仪表盘

凌晨2:14,结账流程正在失败。 到2:15,调查已经在运行。

加入候补名单

告警触发,Polylane接手。 真实还是噪声:一分钟内给出答案。

Issues Elevated error rate on payments-api Timeline

Elevated error rate on payments-api

High Incident ·Detected from a Datadog alert
OverviewMetricsLogsTracesTimelinePropertiesInvestigation
  1. 02:14:07 Alert received

    Datadog · payments-api error rate · occurrence #1

  2. 02:14:09 Triage started

    Fetching the monitor query and metrics 30 min before, 5 min after the firing.

  3. 02:14:58 Triaged as an incident

    Severity high. Error rate 41× the 30-minute baseline, clustered on POST /v1/charges; deploy f3a9c12 landed 18 minutes before the firing.

  4. 02:15:02 Investigation started

    3 hypotheses · 3 passes each

  5. 02:31:44 Alert fired again (occurrence #2)

    Folded into this issue after 16 quiet minutes; the running investigation was nudged.

  6. 03:06:12 Alert recovered

    Provider recovery received. Issue resolved.

它一直挖到能证明为止。 假设经正反双方论证,判定需要证据。

Threads Critical latency degradation in checkout-edge worker Share

What I did

I queued an autofix that restores the Hyperdrive pool hd-prod in the coreplane/checkout-edge repo with:

  • max_connections restored from 5 to 50 on the pool binding
  • a regression test that fails when checkout queues on the pool
  • wrangler.jsonc configured to deploy as checkout-edge

This gives Boris a PR with deployable code. Once merged, running wrangler deploy will replace the degraded worker with the fixed version.

The root cause report has been updated with the occurrence #2 evidence: Root Cause: checkout-edge Latency Degradation (Recurring).

Wall Time
checkout-edge worker · last hour
Export chart
Wall Time P99
28,381ms
Wall Time P95
2,696ms
Wall Time P50
118ms
010k20k30k
18:0018:2519:00
Worked for 4m 12s

AI can make mistakes. Please double check cited sources.

Dig deeper...

故障以修复收尾,而不是以复盘会议收尾。 根因找到,修复写好,等你审查。

github.com/coreplane/payments-api/pull/491

Cap retries on the checkout webhook worker #491

polylane
Open polylane wants to merge 1 commit into main from polylane/autofix/chat/k3x9f2-4e7d21a
Conversation 1 Commits 1 Checks 1 Files changed 2
polylane bot commented 6 minutes ago ···

Retries on the checkout webhook worker were unbounded: a failing delivery re-queued itself forever and amplified load on payments-api. This caps delivery at 5 attempts with exponential backoff and dead-letters the payload after the last one.

What changed

worker/deliver.ts gains MAX_DELIVERY_ATTEMPTS = 5 and backoff between attempts; exhausted payloads land in checkout-webhooks-dlq instead of re-queueing.

Validation

npm test — 214 passed. A forced failing delivery stopped after 5 attempts and appeared in the dead-letter queue.

Root cause · Why it's safe · Out of scope
polylane added commit 4e7d21a Verified
Review required At least 1 approving review is required
ci / test Successful in 3m 12s Details
Review required Waiting on your review: Polylane never merges on its own
Merging is blocked

Polylane如何工作。 它学习你的系统、监视它、调查,然后行动。

  • 它先学习你的系统

    上下文图映射你的云、仓库和可观测性提供商中的每一个资源和依赖。智能体基于真实拓扑推理,而不是猜测。

  • 无阈值检测

    每个提供商都有内置检查,再加上由你自己保存的查询和仪表盘生成的检查。统计分析和智能体共同判断,改善永远不会触发问题。

  • 有据可查的调查

    每一条结论都链接回背后的查询、日志行或变更记录。没有证据的判定回落为无法确定。

  • 写入是争取来的,从不默认拥有

    账户以只读方式连接。回滚默认关闭、受限速并留有记录。代码变更走你正常的审查流程。

  • 它每周都更敏锐

    记忆、每日笔记和用真实数据反复确认的监控查询:七月的调查能从六月的调查中学习。

它接入你已经在运行的一切。 以只读方式连接,然后开始。

常见问题。

Polylane会呼叫我吗?

它不是呼叫系统,也不打算成为一个。Polylane通过邮件、Slack和控制台发送通知,并且只针对它自己检测到的严重和高严重级别的问题。目标是更少的夜间叫醒:问题到达时已经调查完毕。

夜里检测到问题会怎样?

分诊确认它是真实的,调查自动开始,各个假设针对你的真实遥测并行检验。到早上,问题已经有了判定、背后的证据和建议的下一步;在启用了自动修复的地方,修复已经写好并等待审查。

Polylane接入哪些告警来源?

Datadog、Honeycomb、Axiom、Grafana Cloud、Better Stack、Sentry、CloudWatch、Vercel、Render和Cloudflare,再加上一个通用webhook接入其他一切。每个来源用限定范围的遥测令牌进行身份验证,每一次触发都成为一个已分诊的问题。

如果Polylane把真实故障判成噪声怎么办?

“不是故障”是一个判定,而不是删除:问题连同推理过程留在控制台里,你可以一键对任何东西发起调查。判定需要证据,拿不到数据的调查永远不会给出确认。

它会运行多少次调查?

自动调查按滚动24小时设上限:Free套餐10次,付费套餐由你自己设定。手动发起的不计入。定价公开在polylane.com/pricing。

更多使用场景

把夜晚交给智能体。 把早晨留给自己。