How to monitor Cloudflare Durable Objects in production
Explore with AI
Add observability.enabled to the Wrangler config of the Worker that defines your Durable Object class, then use the namespace's Observability tab (filter by $workers.durableObjectId) and Metrics tab, or pull the same metrics from the GraphQL Analytics API. Those cover requests, CPU, memory and storage size. Your own code has to log the rest: errors marked overloaded, retryable or remote that reach the caller, and alarm retry counts.
On this page
- What a Durable Object is, and why that changes monitoring
- Step 1: Turn on logs in the Worker that defines the class
- Step 2: Read the namespace Metrics tab
- Step 3: Pull Durable Object metrics with the GraphQL Analytics API
- Step 4: Log errors at the caller, where they surface
- Step 5: Log alarms, background jobs and storage yourself
- The Durable Object alerts worth having
- Where Durable Object monitoring goes wrong
- Watching Durable Objects with Polylane
- Common questions
Cloudflare gives you three places to watch Durable Objects: the namespace Metrics tab, the namespace Observability tab (Workers Logs), and the GraphQL Analytics API that powers both dashboards. For live debugging, wrangler tail streams console output and exceptions from Durable Object requests as they happen. Turn on all of these first.
The platform can’t tell you everything, though. When a single object gets too busy, the failure shows up as an exception in the calling Worker. Alarm failures are retried quietly, and storage limits only show up when a write fails. So a complete setup has four parts: platform metrics, logs from the class, structured error logs at every call site, and your own logs for alarms and background work.
What a Durable Object is, and why that changes monitoring
A Durable Object is a single instance of a JavaScript class that you deploy to Cloudflare. Each instance has a globally unique ID and its own private, persistent storage, and it handles every request sent to it on one thread. All instances of a class belong to one namespace, which can hold any number of objects. Cloudflare reports most metrics per namespace, while most failures happen to one object. That gap is why monitoring a Durable Object takes more work than monitoring a plain Worker.
Each property of an object creates a failure you have to watch for:
- One object, one thread. Cloudflare’s limits page sets a soft limit of 1,000 requests per second for an individual object. Past that point, requests queue up and the caller gets an overloaded error. Namespace totals can look healthy while one hot object is failing.
- Memory is shared and capped. Each isolate has a 128 MB limit, and one isolate can host several objects of the same class. In-memory state does not survive eviction, hibernation or a crash.
- CPU resets per request. By default an object gets 30 seconds of CPU per incoming request or WebSocket message. You can raise that to 5 minutes with
limits.cpu_ms. If an object uses more than 30 seconds of compute between incoming requests, it is more likely to be evicted and reset. - Errors travel to the caller. Uncaught exceptions, from your code or from Cloudflare’s infrastructure, reach the code that called the stub. Many of them leave that stub broken.
- Alarms retry and then stop. A failing
alarm()handler is retried with exponential backoff, starting at 2 seconds, for up to 6 retries. After that it does not run again until something callssetAlarm(). - Storage has a ceiling. A SQLite-backed object holds up to 10 GB on Workers Paid and 1 GB on Free. At the limit, writes fail with
SQLITE_FULLwhile reads and deletes keep working.
The diagram shows where a request to an object can fail, and which signal records each failure.
graph TB A[Worker calls the object stub] --> B{Object queue too deep?} B -->|yes| C[Caller gets an overloaded error] B -->|no| D[Object runs the request] D --> E{Over CPU or 128 MB memory?} E -->|yes| F[Object reset, remote error to caller] E -->|no| G{Storage write fails?} G -->|yes| H[SQLITE_FULL or storage timeout] G -->|no| I[Response and invocation log] C --> J[Only your caller logs see this] F --> J I --> K[Metrics tab and GraphQL datasets] style C fill:#fee2e2,stroke:#fca5a5,color:#7f1d1d style F fill:#fee2e2,stroke:#fca5a5,color:#7f1d1d style H fill:#fee2e2,stroke:#fca5a5,color:#7f1d1d style J fill:#d1fae5,stroke:#6ee7b7,color:#065f46
Step 1: Turn on logs in the Worker that defines the class
Durable Object logs are grouped by script name and class name. The setting belongs in the Worker that defines the class. Setting it on a Worker that only binds to the class does not give you the object’s logs.
- Open the Wrangler file of the Worker that exports your Durable Object class.
- Add the observability block:
{
"observability": {
"enabled": true,
"head_sampling_rate": 1
}
}
[observability]
enabled = true
head_sampling_rate = 1
- Deploy. You need Wrangler 3.78.6 or later.
npx wrangler --version
npx wrangler deploy
- In the dashboard, go to Durable Objects, pick the namespace, and open the Observability tab.
- To look at one instance, filter on
$workers.durableObjectId. Every Durable Object request log carries the instance ID in that field.
Log JSON objects. Plain strings are hard to search. Workers Logs pulls the fields out of an object and indexes them, so console.log({ room: name, event: "join" }) lets you filter by room. With console.log("room: " + name) you are stuck doing text matches. Each invocation also writes one invocation log, and alarm and RPC invocations show up as their own messages. That means you can see alarm runs without adding any code.
Check the limits before you rely on these logs during an incident. Logs are kept for at most 7 days on Workers Paid and 3 days on Free. Free includes 200,000 log events a day. Paid includes 20 million a month, then $0.60 per additional million. Cloudflare says that from 1 December 2026 Workers Logs moves to Cloudflare Observability pricing. A single log can be up to 256 KB, and anything larger is truncated. To keep logs longer or alert on them, send them out with OpenTelemetry export (Cloudflare’s recommended option), Workers Logpush or a Tail Worker.
For live debugging, run wrangler tail from the project root. It streams console and exception logs for both normal Worker requests and Durable Object requests:
npx wrangler tail
Step 2: Read the namespace Metrics tab
Go to Durable Objects, choose a namespace and open Metrics. The time window defaults to the last 24 hours. You can filter the charts to one object by typing its ID or name, then clear the filter to go back to the whole namespace.
The Memory usage chart needs care. It shows V8 isolate memory as P50, P90, P99 and P999 percentiles, sampled while objects are active. Memory is measured per isolate, and one isolate can host several objects of the same class plus the Worker code around them. When you filter to one object, each sample is still the memory of the whole isolate that hosts it. Use the chart to watch for trends. If memory climbs steadily towards 128 MB, suspect a leak. Take heap snapshots locally with DevTools, or capture a heap profile from production.
When we built observability for Workers at Cloudflare, this per-isolate view was the part people misread most often. A high P99 on a single object can come from a neighbour object sharing the same isolate.
Step 3: Pull Durable Object metrics with the GraphQL Analytics API
The dashboard charts query the GraphQL Analytics API, so you can pull the same numbers into your own dashboards and checks. Four datasets cover Durable Objects:
durableObjectsInvocationsAdaptiveGroups: requests and response sizes per invocation.durableObjectsPeriodicGroups: periodic samples, includingcpuTime, WebSocket message counts andmemoryUsageBytes, with percentiles fromquantiles.memoryUsageBytesP50toquantiles.memoryUsageBytesP999, in bytes.durableObjectsStorageGroups: stored bytes.durableObjectsSubrequestsAdaptiveGroups: subrequests made by your objects.
This query, adapted from Cloudflare’s docs, returns request totals, CPU time and peak stored bytes since a date:
query {
viewer {
accounts(filter: { accountTag: "YOUR_ACCOUNT_ID" }) {
durableObjectsInvocationsAdaptiveGroups(
filter: { date_gt: "2026-10-12" }
limit: 1000
) {
sum { requests responseBodySize }
}
durableObjectsPeriodicGroups(
filter: { date_gt: "2026-10-12" }
limit: 1000
) {
sum { cpuTime }
}
durableObjectsStorageGroups(
filter: { date_gt: "2026-10-12" }
limit: 1000
) {
max { storedBytes }
}
}
}
}
Use GraphQL introspection to find the other fields in each dataset. Cloudflare’s Querying Workers Metrics with GraphQL tutorial covers authentication. Cloudflare also publishes a Grafana dashboard starter for Durable Object metrics, if you want charts without building them yourself.
WebSocket metrics are split across datasets. A WebSocket connection only appears in durableObjectsInvocationsAdaptiveGroups once it closes, and long-lived connections often stay open until the object shuts down. Incoming and outgoing messages are in durableObjectsPeriodicGroups. With the WebSocket Hibernation API, incoming messages appear in durableObjectsInvocationsAdaptiveGroups as well. If a chat app shows almost no requests, it may just be that no connections have closed yet.
Step 4: Log errors at the caller, where they surface
Any exception from an object, whether thrown by your code or by Cloudflare’s infrastructure, reaches the code that called the stub. The exception carries properties that tell you what happened:
.remote: the exception came from your class code. Some infrastructure errors set it too, for example when the object goes over its memory or CPU limit..retryable: a transient error. Retry if the request is safe to repeat, with exponential backoff..overloaded: the object can’t keep up. Do not retry, because retrying makes the overload worse.
Wrap every stub call and log these flags as structured fields. Then you can count overloads per object name:
export async function callRoom(env, name, payload) {
// New stub per call: many exceptions leave a stub broken
const stub = env.ROOM.getByName(name);
try {
return await stub.join(payload);
} catch (e) {
console.error({
event: "do_call_failed",
doClass: "Room",
doName: name,
message: e.message,
retryable: e.retryable === true,
overloaded: e.overloaded === true,
remote: e.remote === true,
});
throw e;
}
}
In Workers Logs you can then filter event = do_call_failed AND overloaded = true and group by doName to find the hot object. These error messages are worth searching for by name, because each one has its own fix:
Durable Object is overloaded.It has four variants: too many requests queued, too much data queued, requests queued for too long, and too many requests for the same object within a 10 second window. The fix is less work per request or fewer requests, often by spreading load across more objects.Your account is generating too much load on Durable Objects.You are creating stubs too fast. It is safe to retry after a short wait.Durable Object reset because its code was updated.A deploy restarted the object. In-memory state is gone. Data already written to storage is safe.Durable Object storage operation exceeded timeout which caused object to be reset.This often comes fromdeleteAll()on a large object. Each call makes progress, so retry until it succeeds.Your account is doing too many concurrent storage operations.Back off, and batch reads withstorage.get([keys]).
Step 5: Log alarms, background jobs and storage yourself
The platform records that an alarm ran. It does not tell you that the handler has used up its retries. The alarm() handler receives alarmInfo with retryCount and isRetry, so log both. Then catch the error and reschedule, so a long outage in a downstream service can’t use up all 6 retries:
async alarm(alarmInfo) {
const retryCount = alarmInfo?.retryCount ?? 0;
console.log({ event: "alarm_start", objectId: this.ctx.id.toString(), retryCount });
try {
await this.processDueEvents();
console.log({ event: "alarm_ok", objectId: this.ctx.id.toString() });
} catch (e) {
console.error({ event: "alarm_failed", objectId: this.ctx.id.toString(), retryCount, message: e.message });
// Reschedule so retries never run out
await this.ctx.storage.setAlarm(Date.now() + 60_000);
}
}
An alarm handler can run for at most 15 minutes of wall time per invocation. For long jobs, save progress after every step and log it. Cloudflare provides no shutdown hook, so a job that stopped halfway looks the same as one still running unless it records its own progress.
For storage, catch SQLITE_FULL around writes and log it with the object ID. Watch storedBytes from durableObjectsStorageGroups to see growth before you reach the limit. To look inside one object’s storage, Data Studio (beta) in the dashboard lets you view and edit SQLite-backed objects by name or ID. It needs at least the Workers Platform Admin role, every query is written to the audit log, and queries count towards normal usage billing.
The Durable Object alerts worth having
Workers Logs is for searching and the Metrics tab is for looking. To get paged, send logs out with OpenTelemetry export, Logpush or a Tail Worker, or run the GraphQL query on a schedule. Then alert on these:
- Overloaded errors by object name. Any steady rate of
overloaded = truemeans one object is a bottleneck. - Memory P99 trending towards 128 MB in
durableObjectsPeriodicGroups. A steady climb points to a leak. - CPU time per request close to your
cpu_mssetting, 30 seconds by default. alarm_failedevents orretryCountabove zero. A handful is noise. A rising count means something downstream is broken.SQLITE_FULL, orstoredBytesfor an object nearing 10 GB on Paid or 1 GB on Free.- Reset errors after a deploy. A burst of
reset because its code was updatedis expected. A long tail of them means clients are not reconnecting or retrying. - Account-level load errors, which point to stub creation or storage concurrency in your code.
Where Durable Object monitoring goes wrong
- Logs turned on in the wrong Worker. The Observability tab stays empty because
observability.enabledwas set on the caller and not on the Worker that defines the class. Fix: add it to the defining Worker and redeploy. - Reading the memory chart as one object’s memory. It always shows the whole isolate’s memory. Fix: use it for trends and confirm with a heap profile.
- Head sampling hiding rare objects. At a
head_sampling_rateof 0.01, only one request in a hundred is logged, so a single failing object can disappear from view. Fix: keep sampling at 1 for Durable Object classes, or log errors from the caller where you control the volume. Also note that past 5 billion logs per account per day, Cloudflare applies a 1% sample for the rest of the day. - Retrying overloaded errors. Generic retry middleware turns an overloaded object into an outage. Fix: check
.overloadedand fail fast. - Reusing a broken stub. After an exception, many stubs fail straight away with the same error, which inflates your error counts. Fix: create a new stub for each attempt.
- Missing WebSocket traffic. Connections only count once they close. Fix: read message counts from
durableObjectsPeriodicGroups. - Alarms that silently stop. After 6 failed retries, nothing runs. Fix: catch errors and reschedule, then alert on
alarm_failed. - Short retention. Logs are kept for 7 days at most. Fix: export to a tool that keeps them longer if your incident reviews need more history.
Watching Durable Objects with Polylane
Polylane detects production issues and fixes them by opening pull requests. It connects to Cloudflare, so it can see when a Durable Object starts failing. It works out why, and the fix arrives ready for your team’s normal review.
Running on Cloudflare? See how Polylane monitors Cloudflare in production.
Common questions.
Where do I find Durable Object logs in the Cloudflare dashboard?
Go to Durable Objects, select the namespace and open the Observability tab. Logs only appear after you add observability.enabled = true to the Wrangler config of the Worker that defines the class and redeploy. To see one instance, filter on $workers.durableObjectId.
Can I see metrics for a single Durable Object instance?
Yes. On the namespace Metrics tab, type the object's ID or name in the filter and select a match. Memory usage is still the whole isolate's memory, which can include other objects of the same class, so treat it as a trend line for that object.
Which GraphQL datasets hold Durable Object metrics?
There are four: durableObjectsInvocationsAdaptiveGroups, durableObjectsPeriodicGroups, durableObjectsStorageGroups and durableObjectsSubrequestsAdaptiveGroups. Memory percentiles come from durableObjectsPeriodicGroups as quantiles.memoryUsageBytesP50 to P999, and storage size comes from storedBytes in durableObjectsStorageGroups.
How long does Cloudflare keep Durable Object logs?
Workers Logs keeps logs for up to 7 days on Workers Paid and 3 days on Free. Paid includes 20 million log events a month, then $0.60 per additional million, and Free includes 200,000 events a day. To keep logs longer, export them with OpenTelemetry export, Logpush or a Tail Worker.
How do I know a Durable Object is overloaded?
The calling Worker gets an exception whose message starts with Durable Object is overloaded and whose .overloaded property is true. Log that flag with the object name at every call site so you can find the hot object. Do not retry these errors. Spread the load across more objects or do less work per request.
Why do my WebSocket Durable Objects show so few requests?
A WebSocket connection is only counted in durableObjectsInvocationsAdaptiveGroups once it closes, and long-lived connections often stay open until the object shuts down. Read incoming and outgoing message counts from durableObjectsPeriodicGroups. With WebSocket Hibernation, incoming messages also appear in durableObjectsInvocationsAdaptiveGroups.
How do I monitor Durable Object alarms?
Log the retryCount and isRetry values that alarm() receives, and log a failure event when the handler throws. Failed alarms are retried up to 6 times with exponential backoff starting at 2 seconds, and then stop. So catch errors, call setAlarm() again, and alert on the failure event.
Sources (10)
- Durable Objects metrics and analytics (Cloudflare Docs)
- Durable Objects troubleshooting (Cloudflare Docs)
- Durable Objects error handling (Cloudflare Docs)
- Durable Objects limits (Cloudflare Docs)
- Durable Objects alarms (Cloudflare Docs)
- Workers Logs (Cloudflare Docs)
- Cloudflare changelog: Data Studio for Durable Objects
- Durable Objects now Generally Available (Cloudflare Blog)
- Cloudflare Durable Objects pending I/O keep-alive guide (Polylane)
- Polylane for Cloudflare
Boris Tane is the founder of Polylane. He previously founded Baselime, observability for the future of the cloud, which Cloudflare acquired. At Cloudflare he built and led the Workers observability team.
More in this series
- 1 How to Monitor a Django App on Render
- 2 How to Monitor a FastAPI App on Railway: Logs, Traces and Alerts
- 3 How to Monitor a Supabase App in Production
- 4 How to Debug Cloudflare Workers Errors: Logs, Traces and Error Codes
- 5 How to debug Vercel function timeouts
- 6 Vercel 504 Gateway Timeout on Serverless Functions: Causes and Fixes
- 7 Cloudflare Workers error 1101: causes and how to fix it
- 8 Cloudflare Hyperdrive connection errors: causes and fixes
- 9 How to Monitor a Convex App in Production
- 10 How to monitor Cloudflare Durable Objects in production
- 11 How to roll back a Cloudflare Workers deployment
- 12 How to trace requests across Cloudflare Workers and service bindings
- 13 How to Monitor Vercel Cron Jobs: Heartbeats, Logs and Alerts
- 14 How to reduce Vercel Function cold starts
- 15 How to Monitor a Next.js App on Render
- 16 How to debug Modal function timeouts
- 17 Modal container out of memory: find the cause and fix it
- 18 Fix Railway “Healthcheck failed!” on deploy
- 19 How to Search Railway Logs for Errors: Filters, CLI and HTTP Logs
Related
- Cloudflare Durable Objects pending I/O keep-alive guide From 2026-10-01, pending I/O keeps Cloudflare Durable Objects in memory after the client leaves. What counts, the 15-minute limit, flags and billing.
- Fix “Durable Object reset because its code was updated” Why a deploy makes Cloudflare Durable Objects throw “reset because its code was updated”, and how to retry safely, keep clients connected and lose no state.
- Profiling Cloudflare Workers CPU and memory with flamegraphs Capture on-demand CPU and heap profiles of deployed Cloudflare Workers and Durable Objects, read the flamegraph, and avoid the limits that trip teams up.
- Workers Issues: route Cloudflare errors to coding agents Cloudflare's new Workers Issues groups exceptions and 5xx errors and sends them to Claude Code, Cursor, Devin or a webhook. Setup, automations and limits.
- Fix Cloudflare “Error 1102: Worker exceeded resource limits” Why a Cloudflare Worker returns Error 1102, how to tell a CPU time overrun from a 128 MB memory overrun, and the code and Wrangler changes that fix each.