儀表板

一天十次部署。其中一次是壞的。 Polylane 找到它,並把它復原。

加入候補名單

每一次部署開箱即受監看。 改了什麼、誰改的,以及最可能出問題的訊號。

Clouds prod-aws Changes

SQS visibility timeout lowered on checkout-events

Moderate impact configuration ·Synced 12 minutes ago

VisibilityTimeout on checkout-events dropped from 120s to 15s. Consumers that hold a message longer than 15 seconds will see it delivered twice; the dead-letter queue threshold is unchanged.

What we're watching
ApproximateAgeOfOldestMessage
checkout-events · baseline at change 3.2s · worse if up
Watching: no issue since this change
NumberOfMessagesReceived
checkout-events · baseline at change 41/min · worse if up
Watching: no issue since this change
Triggering events
SetQueueAttributes CloudTrail · deploy-bot · 12:41:02Z attached to this record's delta window
Full diff (3 changes)
Nodes (2) Edges (1)
2 nodes modified · 1 edge removed

復原,如果你想要。 預設關閉。打開它,剩下的由代理處理。

Issues Critical latency degradation in checkout-edge worker Overview

Critical latency degradation in checkout-edge worker

Incident critical Polylane ·Detected 3 hours ago ·Last seen 4 minutes ago ·2 occurrences
OverviewInvestigationMetricsTimelineProperties
checkout-edge Cloudflare Worker

Critical latency degradation detected in checkout-edge worker: 18x+ P99 latency spikes sustained for 12 minutes

Causal metrics All metrics
Request Duration
414 ms ▲ 43.1σ
Wall Time
416 ms ▲ 43.0σ
CPU Time
14 ms ▲ 2.6σ
Blast radius
Search your cloud resources...
Graph Table
edge-gateway Cloudflare Worker ··· checkout-edge Cloudflare Worker ··· hd-prod Hyperdrive ··· payments-db PlanetScale ··· cart-svc Cloudflare Worker ···
Analysis Copy

Deploy 9f3c2a1 shrank the Hyperdrive pool hd-prod from 50 connections to 5. Under checkout load, requests queue on connection checkout and P99 rises 18× against the 30-minute baseline. Restoring the pool size restores latency.

Tags
service · checkout-edgeprovider · cloudflaredeploy · 9f3c2a1signal · wall_time_p99

現在復原。之後好好修。 復原爭取時間:真正的修復隨後送到你面前審查。

github.com/coreplane/payments-api/pull/491

Cap retries on the checkout webhook worker #491

polylane
Open polylane wants to merge 1 commit into main from polylane/autofix/chat/k3x9f2-4e7d21a
Conversation 1 Commits 1 Checks 1 Files changed 2
polylane bot commented 6 minutes ago ···

Retries on the checkout webhook worker were unbounded: a failing delivery re-queued itself forever and amplified load on payments-api. This caps delivery at 5 attempts with exponential backoff and dead-letters the payload after the last one.

What changed

worker/deliver.ts gains MAX_DELIVERY_ATTEMPTS = 5 and backoff between attempts; exhausted payloads land in checkout-webhooks-dlq instead of re-queueing.

Validation

npm test — 214 passed. A forced failing delivery stopped after 5 attempts and appeared in the dead-letter queue.

Root cause · Why it's safe · Out of scope
polylane added commit 4e7d21a Verified
Review required At least 1 approving review is required
ci / test Successful in 3m 12s Details
Review required Waiting on your review: Polylane never merges on its own
Merging is blocked

Polylane 如何運作。 它學習你的系統、監看它、調查,然後行動。

  • 它先學習你的系統

    情境圖描繪出你的雲端、儲存庫與可觀測性供應商之間的每一個資源與依賴關係。代理依據真實的拓樸推理,而不是猜測。

  • 不靠閾值的偵測

    每個供應商都有內建檢查,再加上從你自己儲存的查詢與儀表板產生的檢查。一次統計分析與一個代理共同決定,而改善永遠不會引發問題。

  • 拿得出證據的調查

    每一項主張都連回背後的查詢、日誌行或變更記錄。沒有證據的判定會退回無法定論。

  • 寫入是掙來的,絕非理所當然

    帳戶以唯讀方式連接。復原預設關閉、受頻率限制,且留有記錄。程式碼變更走你平常的審查流程。

  • 它每週都更敏銳

    記憶、每日筆記與對照真實資料重新確認的監控查詢:七月的調查會從六月的學到東西。

它接上你已經在跑的一切。 以唯讀方式連接,然後開始。

常見問題。

Polylane 能復原哪些平台?

Cloudflare、Vercel、Render 和 Fly.io 的部署,還原到最後一個已知正常的部署。復原預設關閉:由你自己啟用,也可以隨時關閉。

什麼能阻止自動復原迴圈?

平台裡的硬性限制,而不是代理的判斷。每個目標每小時最多三次成功復原、同一時間只有一次復原在進行,而當平行分析對要還原的版本意見不一致時,動作會被擋下並升級給你。

它需要我的 CI 管線嗎?

不需要。Polylane 直接從供應商讀取部署。pull request 審查以你可以設為必要的 GitHub 檢查執行,但你的管線不需要任何改變。

那些不是部署的變更呢?

設定編輯、擴縮事件與安全性變更也以同樣的方式記錄與監看。當某個佇列在有人調低它的可見性逾時十二分鐘後開始異常,調查就從那次變更開始。

我能看到復原為什麼發生嗎?

每一次復原都留有記錄:執行它的那次執行、退化的訊號、還原的版本與推理過程,都會寫進記錄。

更多使用情境

星期五也放心合併。 週末交給 Polylane。