What is change intelligence? A production safety guide

What is change intelligence? Change intelligence is the practice of connecting production changes to system behavior, so you can identify what changed, where it took effect, and whether it contributed to an incident. It covers code deploys, configuration edits, feature flags, infrastructure changes and dependency updates. The useful output is an evidence-backed explanation, not a list of commits near an alert.

Consider a checkout failure. The application release might be unchanged, but a flag could have enabled a different payment path, a timeout could have been reduced, or a dependency could have changed its response. A deploy-only history cannot answer all three questions. A change timeline should show those possibilities next to the failing requests and the resources they touched.

Why look for changes when an incident starts?

There is published evidence for starting here. In Google’s analysis of thousands of postmortems from 2010–2017, binary pushes accounted for 37% and configuration pushes for 31% of the listed outage triggers. Those are findings from Google’s historical sample, not a measured rate for your application or the whole industry. The appendix also separates triggers from contributing root causes: the change that exposed a defect is not necessarily the only thing that needs fixing.

The Google SRE Workbook’s release chapter explains the mechanism. Tests and test environments do not cover every production input or state. New code, dependency behavior and configuration can expose defects only when real traffic arrives. Feature flags also separate feature launches from binary releases: deploying code and enabling its behavior are distinct changes.

This makes recent changes a useful first hypothesis. It does not justify ignoring traffic shifts, resource exhaustion or failures elsewhere. A timeline should help reject irrelevant changes as readily as identify a plausible one.

What a useful change record tracks

The following is a recommended record design, not a universal provider schema. Give each event enough context that an on-call engineer can test a hypothesis without asking its author to reconstruct the release.

Record the source event ID, actor when available, source URL, event time and collection time too. Keep unknown fields unknown. Record secret rotation identifiers rather than secret values. The person or automation that made a change is useful for finding context, not assigning blame.

Distinguish an edit from its application. Google’s configuration guidance recommends logging both configuration changes and their application because a committed configuration may not yet be applied. An approved pull request, uploaded artifact and serving release are three different observations.

Platform boundaries matter. Cloudflare Workers versions capture code, assets, bindings and compatibility settings, while deployments determine the versions serving traffic. Associated storage state is not tracked by those versions. In Kubernetes, changing a Deployment’s Pod template triggers a rollout; scaling it does not. Both scaling and rollout can belong on an operational timeline even though only one creates a new deployment revision.

Change intelligence versus neighboring practices

These are working distinctions for organizing your response process. Teams and vendors use the terminology differently; judge a setup by the questions it can answer.

Change management asks whether a change should proceed and under what controls. Its records include review, authorization and scheduling. Change intelligence asks what happened after those decisions. An approved change can still cause a failure, and an emergency dashboard edit may never have passed through the normal review workflow.

Observability supplies the evidence about system behavior. Google’s monitoring chapter distinguishes symptoms from causes and describes retrospective analysis as asking what else happened when latency rose. Change intelligence brings the change records into that analysis. Without telemetry, a change history cannot tell you whether users were affected. Without change context, a graph alone cannot identify the responsible edit.

Deployment tracking records the progress of releases. It is a necessary input, but a successful deploy status says little about later flag edits, routing changes or an external API. Change intelligence connects deployment tracking to those other changes and to incident evidence.

Delivery performance measurement summarizes outcomes over time. DORA defines change fail rate as the ratio of deployments requiring immediate intervention, and failed deployment recovery time as the time to recover from such a deployment. Neither is the percentage of all incidents caused by changes. Individual incident attribution can inform those measurements, but proximity alone should not label a deploy as failed.

Correlation is the beginning of the investigation

A good change hypothesis passes several checks:

1. Order: the change took effect before the relevant failures began. Use exposure time, not just commit time.
2. Scope: the failing service, region or cohort actually received the change.
3. Mechanism: the diff or setting explains the observed exception, delay or rejected request.
4. Comparison: an unchanged version or unexposed cohort behaves differently under comparable conditions.
5. Recovery: stopping or reversing the suspected change removes the symptom, when that action is appropriate and reversible.

These checks are a practical investigation framework. The SRE canary chapter provides the underlying comparison method: compare a changed population with a control, and account for shared dependencies. It also warns that before/after comparisons are vulnerable to time-dependent factors. If both populations fail because they share a backend, the new release is not exonerated or convicted by their similarity alone.

For example, suppose only requests with a new flag enabled fail. The binary was deployed earlier without errors. A flag event, per-request flag exposure and a log showing failure in the newly enabled branch make a stronger case than the binary’s deploy timestamp. If turning the flag off restores the route, retain that observation alongside the code defect it exposed.

Start with tools you already operate

Pick one user-facing service rather than ingesting every audit stream at once. Agree on one service name and environment name across its deployment records, logs and dashboard. Save the artifact identity and the time it began serving traffic for the next release.

Then put that marker on the service’s health charts. Grafana annotations support event descriptions, tags and links, and can be created manually, via the HTTP API or from annotation queries. A manual marker with a release link is a reasonable first step. Make the next step automation from the deploy system so the record does not depend on someone remembering during an incident.

Add configuration and flag changes next. Test one harmless change through the entire path: source record, stored event, dashboard marker and log filter. Write down any blind spot, such as an external provider with no event feed. “No changes found” means little unless you know which sources were checked.

The companion guide on how to correlate incidents with deploys walks through platform events and a worked investigation. For a Workers application, compare the Cloudflare monitoring overview with the Cloudflare integration’s available context. When evaluating a change intelligence tool, ask whether its conclusions link back to the original change records and request evidence.

What a mature setup adds

A mature setup should preserve a reconstructable history, not just decorate a dashboard. These are useful acceptance criteria:

• Coverage: code, flags, configuration, infrastructure and dependencies have named sources and documented gaps.
• Identity: each change maps to an environment and resource; dependency relationships narrow which changes could affect an incident.
• Exposure: staged rollouts record their boundaries, so a responder can distinguish uploaded, available and serving versions.
• Reliable collection: source IDs support deduplication, delayed arrivals retain original timestamps, and collection failures are visible.
• Evidence: a hypothesis links the change diff to representative failing requests and comparable healthy traffic.
• Retention: records and incident evidence remain available long enough for your postmortem process.
• Learning: confirmed contributing changes and rejected hypotheses feed back into tests, review and rollout practices.

Use real incidents to judge completeness. Can a teammate reproduce your conclusion from the saved records? Could they discover that a flag changed without a deploy? Can they see when collection was unavailable? These questions are more useful than counting integrations.

For concrete consequences of deployment boundaries, see the guide to Durable Objects resetting after a code update. For the controls before a change reaches production, see keeping AI coding agents from breaking production. Change intelligence completes that loop by making the production outcome inspectable.

Running on Cloudflare? See how Polylane monitors Cloudflare in production.