Ambrook

Ambrook merged 30 production fixes from Polylane in its first two weeks

In its first two weeks with Polylane, Ambrook merged 30 fixes it didn't have to write, and caught 10 risky changes before they shipped. Polylane watches 1,478 cloud resources on Google Cloud and Cloudflare, proves the cause of each problem and opens the pull request, and about three in four of Ambrook's engineers now work with it.

1,478 cloud resources monitored on Google Cloud and Cloudflare
30 Polylane pull requests merged in its first two weeks
1,121 alerts triaged in its first two weeks
75% of Ambrook's engineering team using Polylane

Connected to Polylane

The challenge

Ambrook handles the books and the payments of the businesses that use it. A change that breaks a sync, a report or a payment shows up in a customer's accounts, and a farm closing out its month can't wait for an engineer to find it.

Ambrook runs on Google Cloud and Cloudflare, with a web and a mobile app on top. Every alert from Sentry and Honeycomb needed someone to work out what was behind it.

“Our customers run their farms and other real economy businesses on Ambrook, so a broken sync or payment lands in someone's books. Polylane finds those problems, works out why they happened and opens the fix. Most mornings there's a pull request waiting that we just review and merge.”
Adam Markon Engineering at Ambrook

How Ambrook uses Polylane

Ambrook connected Google Cloud, Cloudflare, GitHub, Sentry, Honeycomb and Slack on its first day, and Linear a week later. Polylane mapped 1,478 cloud resources into a context graph and tied them to the repositories that deploy them.

From there it works on its own. It watches production and the alerts from Sentry and Honeycomb, folds repeats into one issue, and investigates the ones that matter until it can prove the cause. When the cause is in Ambrook's code, it writes the fix, reviews it, and opens a pull request with the evidence. The team reviews it like any other change and merges it.

Within four weeks of starting, three in four of Ambrook's engineers were using it.

0 50 100 150 200 20 Oct 28 Oct 196
Polylane triaged 1,121 alerts for Ambrook in its first two weeks
Alerts from Sentry, Google Cloud, GitHub Actions and Linear that Polylane triaged each day, up to 196 in a single day.

On call, with Polylane

Ambrook's on-call engineer spends the shift with Polylane instead of jumping between Sentry, Honeycomb and Google Cloud to piece an alert together. Polylane has already done the cross-referencing, and the fix is often waiting.

More than once, the fix a teammate shared in Slack turned out to be one Polylane had found. Each one comes with a short video that walks through what broke and why.

“Polylane is seriously saving me so much time and headache and cross-referencing the different platforms we have alerts from.”
Laura Lu Engineering at Ambrook

Stopping risky changes before they ship

Polylane checks every pull request against the live services it deploys to. In the first two weeks it reviewed 2,170 of them and caught 10 changes that would have put production at risk, before they merged.

0 1 2 3 4 20 Oct 28 Oct 3
10 risky changes caught in the first two weeks
Pull requests Polylane flagged as a risk to production, by day.

The results

In its first two weeks, Ambrook merged 30 fixes from Polylane. Most came from problems nobody had filed: Polylane found them by exploring Ambrook's code, from its alerts and from its own monitoring of production.

Several were in the accounting core, where correctness matters most, from contention under load to background work that ran twice. Others were bugs customers would have run into, security hardening, and vulnerable dependencies brought up to date.

Others fixed builds and deploys that had broken. Polylane read the failing run, found the cause and opened the fix, so nobody on the team had to dig through it.

In those two weeks, Polylane triaged 1,121 alerts, detected 1,616 issues and folded 380 repeat detections into issues that were already open, so the team saw each problem once.

0 10 20 30 40 20 Oct 28 Oct 30
Fixes Ambrook merged in its first two weeks
A running total, by the day each fix merged.

Nobody should be on-call.

Related customer stories