Cloudflare Hyperdrive connection errors: causes and fixes

A Hyperdrive connection error tells you which of three hops failed. There is the hop from Hyperdrive to your database, the pool of origin connections Hyperdrive shares across your Workers, and the database client in your Worker code. Each hop fails with its own messages. Once you know which one you’re looking at, the fix is usually one change.

This page lists every message Cloudflare documents for those hops. It shows how to confirm which cause is yours with the pool metrics and a quick database query, gives the fix for each, and covers the alerts that stop the errors coming back.

Where a Hyperdrive connection error comes from

Hyperdrive sits between your Worker and a regional Postgres or MySQL database. Your driver connects to Hyperdrive at the edge, next to your Worker, which is fast. Hyperdrive then runs the query over a pooled connection that sits close to your database. That saves the seven round trips a fresh connection needs: one for the TCP handshake, three for TLS and three for authentication. When the request ends, the Worker’s client is garbage collected, and the origin connection stays in the pool for the next request.

That design gives you three kinds of failure:

1. Configuration errors. When you create or update a config, Hyperdrive runs a test connection and an empty test query (a ; in PostgreSQL). If that fails, you get a numbered code from 2008 to 2016, or the message Failed to connect to the provided database.
2. Runtime errors from Hyperdrive. These arrive as ErrorResponse wire protocol messages, so your driver throws them from the query that caused them. Hyperdrive errors with no matching PostgreSQL code use the code 58000. Errors sent by your database pass through unchanged where possible.
3. Errors from your own code. These are Workers runtime and driver errors, and they almost always mean a client was created in global scope or kept for another request.

Configuration codes 2008 to 2016: Hyperdrive can’t reach or sign in to the database

These appear when you create or update a config. Each code points at one cause.

Hyperdrive configuration error codes and their fixes

Code	What Hyperdrive found	Fix
2008	Bad hostname, it could not be resolved	Confirm the hostname exists in public DNS
2009	Hostname does not resolve to a public IP	Use a public address, or connect privately through Workers VPC or Tunnel
2010	Cannot connect to host and port	Check the spelling and the public DNS record
2011	Connection refused	Allow connections from Cloudflare in your firewall or ACL
2012	Database does not support TLS	Turn on TLS, Hyperdrive requires it
2013	Invalid credentials	Check the username exists and the password case
2014	Database name does not exist	Use the database name, and check it exists
2015	Generic failure with no known reason	Open a Cloudflare support ticket
2016	Test query failed	Grant the user read and write permissions

Code 2010 is a routing problem: check that the hostname has a public DNS record that resolves to a public IP address, and that it isn’t misspelled. Code 2011 is the firewall code. Hyperdrive connects from Cloudflare’s IP address ranges, which all Hyperdrive configurations share with other Cloudflare products. Cloudflare documents three ways to let it in:

1. Allow inbound connections from the whole public Internet.
2. Allow the public Internet, with an IP access control list that permits only Cloudflare’s ranges.
3. Keep the database on a private network and run a Cloudflare Tunnel inside that network. Workers VPC is the other private option in the docs.

If you see Failed to connect to the provided database and none of the codes, look at certificates. Server return error and closed connection means the database demands client certificates and your config doesn’t send them. TLS handshake failed: cert validation failed means you gave Hyperdrive a server CA certificate that didn’t sign the certificate the database presented. Either the CA is wrong, or you are pointing at the wrong database.

Failed to acquire a connection from the pool

This is the runtime error most teams mean when they search for Hyperdrive connection errors. Hyperdrive waited for a free origin connection and timed out, or it couldn’t connect at all. If it happens now and then, the pool is being exhausted: too many connections are held for too long.

Why the pool runs dry

Hyperdrive pools in transaction mode. A client holds one origin connection for the whole of a transaction and gives it back when the transaction ends. Hyperdrive doesn’t limit client connections from your Workers, but it does cap origin connections per configuration:

• Free plan: about 20 origin connections per configuration.
• Paid plan: about 100 origin connections per configuration.
• Every configuration: at least 5.

The usual causes, in the order worth checking:

• Long transactions. Each open transaction holds a connection. An external API call or heavy computation between BEGIN and COMMIT keeps that connection away from every other Worker isolate.
• Slow queries. A query can run for up to 60 seconds before Hyperdrive kills it. A handful of those at once fills a small pool.
• Durable Objects that keep a client open. A Durable Object keeps state across requests. If it keeps a database client open, that connection stays allocated from the pool, and many long-lived objects doing the same can drain it.
• Transactions used to keep SET state. Hyperdrive runs RESET on a connection when it goes back to the pool. Wrapping many operations in one transaction to keep a SET alive holds the connection the whole time. Put the SET in each query or transaction that needs it.
• A pool sized too small. Someone lowered the origin connection limit, or traffic grew past it.

Confirm it with the pool metrics

Hyperdrive exports pool size metrics through the hyperdrivePoolSizesAdaptiveGroups dataset in the GraphQL Analytics API. You can also see them in the Pool connections chart on the Metrics tab of each configuration. Run this query for the window when the errors started:

query HyperdrivePoolSizes(
  $accountTag: string!
  $configId: string!
  $datetimeStart: Time!
  $datetimeEnd: Time!
) {
  viewer {
    accounts(filter: { accountTag: $accountTag }) {
      hyperdrivePoolSizesAdaptiveGroups(
        limit: 10000
        filter: {
          configId: $configId
          datetime_geq: $datetimeStart
          datetime_leq: $datetimeEnd
        }
      ) {
        avg { currentPoolSize availablePoolSlots waitingClients }
        max { maxPoolSize currentPoolSize waitingClients }
        dimensions { coloCode }
      }
    }
  }
}

Read it like this:

• Waiting clients spike, and peak open connections sit at the pool maximum. The pool is exhausted. Go on to the database check below.
• Open connections are well under the maximum, but errors continue. A likely explanation is that Hyperdrive can’t open new connections. Look at the next section on connection_refused.
• One coloCode is bad and the others are fine. The pressure is in one location’s pool. Hyperdrive is distributed, so a client that can’t reach an existing pool gets a new pool with its own allowance. Connection counts can then briefly go over the listed limits.

Metrics are kept for 31 days, so you can compare the incident with a normal week.

Find what is holding the connections

On Postgres, list the sessions that have been busy or idle inside a transaction for a long time:

select pid, usename, state,
       now() - xact_start as transaction_age,
       left(query, 120) as query
from pg_stat_activity
where datname = current_database()
  and state in ('active', 'idle in transaction')
  and xact_start is not null
order by xact_start
limit 20;

Rows in the idle in transaction state are the ones to chase first. Some code opened a transaction and went off to do something else.

Fix the pool exhaustion

1. Shorten the transactions. Do only the essential queries inside the transaction. Move API calls and computation outside it:

// Before: the connection is held while the payment API responds
await sql.begin(async (tx) => {
  const [order] = await tx`select * from orders where id = ${id} for update`;
  const charge = await fetch(env.PAYMENTS_URL, { method: "POST", body: JSON.stringify(order) });
  await tx`update orders set status = 'paid' where id = ${id}`;
});

// After: call the API first, then hold the connection for two quick statements
const charge = await fetch(env.PAYMENTS_URL, { method: "POST", body: JSON.stringify({ id }) });
await sql.begin(async (tx) => {
  await tx`select 1 from orders where id = ${id} for update`;
  await tx`update orders set status = 'paid' where id = ${id}`;
});

2. Close Durable Object clients when they sit idle. Use connection timeouts, and limit how many objects hold a database connection at all.
3. Raise the origin connection limit, carefully. Change it in the dashboard under Settings, then Origin connection limit, or with Wrangler:

npx wrangler hyperdrive update $HYPERDRIVE_ID --origin-connection-limit=60

The limit is soft. During network trouble, Hyperdrive may open more connections than you set, so keep it below your database’s own maximum. If several Hyperdrive configurations point at the same database, add their limits together. If open connections keep reaching the plan limit, Cloudflare takes limit increase requests through a form linked from the limits page.

Server connection attempt failed: connection_refused

This runtime error means Hyperdrive can’t open new connections to your origin. There are two causes. The first is a firewall or ACL rejecting Cloudflare, often after someone tightened the rules or the database moved. The second is your database provider refusing connections because you went over its connection limit.

To tell them apart, check the database’s own connection count at the time of the errors. If it was at the provider’s maximum, the soft pool limit, other clients sharing the database, or several Hyperdrive configs filled it. Lower the Hyperdrive limit, or raise the database’s. If the count was low, fix the firewall with one of the three networking options above.

Errors that mean a client outlived its request

Workers don’t allow I/O across requests. A client created in global scope, or cached in a variable and reused, belongs to a request context that has ended. The messages depend on the driver:

• Workers runtime: Disallowed operation called within global scope (connecting during script startup) and Cannot perform I/O on behalf of a different request.
• node-postgres: Connection terminated, Connection terminated unexpectedly, Client has encountered a connection error and is not queryable, Client was closed and is not queryable, Cannot use a pool after calling end on the pool, Client has already been connected. You cannot reuse a client.
• Postgres.js: write CONNECTION_ENDED, write CONNECTION_DESTROYED and write CONNECTION_CLOSED, followed by the host and port. The code property on the error holds the code.
• mysql2: Can't add new command when connection is in closed state, Connection lost: The server closed the connection. (PROTOCOL_CONNECTION_LOST), Pool is closed.
• mysql: PROTOCOL_ENQUEUE_AFTER_FATAL_ERROR, PROTOCOL_ENQUEUE_AFTER_QUIT and PROTOCOL_ENQUEUE_HANDSHAKE_TWICE.

The fix is the same for all of them: create the client inside the handler, on every request. Don’t create a driver-level pool (new Pool() or createPool()) in global scope. Hyperdrive already pools for you, so a new client per request is fast.

import { Client } from "pg";

export default {
  async fetch(request, env, ctx): Promise<Response> {
    const client = new Client({ connectionString: env.HYPERDRIVE.connectionString });
    await client.connect();
    const result = await client.query("SELECT id, status FROM orders LIMIT 10");
    // No client.end() needed: the client is cleaned up when the request ends,
    // and the origin connection stays in Hyperdrive's pool.
    return Response.json(result.rows);
  },
} satisfies ExportedHandler<Env>;

You don’t need to call client.end(), sql.end() or connection.end(). Clients are cleaned up when the request or invocation ends, including when a Workflow or Queue consumer finishes, or when a Durable Object hibernates or is evicted.

Driver errors that look like connection failures

A few errors show up while you connect, but they come from the driver or the runtime:

• Uncaught Error: No such module "node:...": your code or a library needs a Node module. Turn on Node.js compatibility for the Worker.
• Code generation from strings disallowed for this context: the driver calls eval(), which Workers don’t support. This is common with mysql2. Configure the driver not to use it.
• Hyperdrive does not currently support MySQL COM_STMT_PREPARE messages: remove prepared statements from your MySQL queries.
• Internal error.: something broke on Cloudflare’s side. Check for an ongoing Hyperdrive incident, contact support, and retry if retries are safe for your workload.

Recovering after a database failover

Hyperdrive detects and recovers from most failovers by itself. If connections still point at the old primary afterwards, restart the pool. Open the configuration in the dashboard, go to Settings, and select Restart under Danger zone. You need the Hyperdrive Admin role. A restart drops every active connection and rebuilds the pool, so in-flight queries may see brief errors. Do it once, then watch the pool chart refill.

Reproducing the error locally without fooling yourself

Plain wrangler dev connects straight to the database through localConnectionString, or through the CLOUDFLARE_HYPERDRIVE_LOCAL_CONNECTION_STRING_ variable followed by your binding name. In that mode, pooling and caching don’t take effect, so pool exhaustion and Hyperdrive’s own errors won’t reproduce.

wrangler dev --remote runs your Worker on Cloudflare with the deployed Hyperdrive config, so pooling and caching are active. Any writes land in the configured database, which may be production. Point a separate Hyperdrive config at a staging database before you load test pool behaviour.

Stale reads are a caching issue

If a read after a write returns old data, the connection is fine and the query cache is serving the read. Hyperdrive doesn’t invalidate cached reads when you write. Check cacheStatus in the hyperdriveQueriesAdaptiveGroups dataset. For reads that must be fresh, such as sessions, permissions and reads right after a write, add a second binding to the same database with caching turned off. It still gets pooling and fast connection setup.

What keeps Hyperdrive connection errors coming back

• Raising the limit to cover a leak. A bigger pool buys time until the idle transactions fill it again. Fix the transaction first, then size the pool.
• Setting the Hyperdrive limit equal to the database maximum. The soft limit lets Hyperdrive go over it, which turns into connection_refused. Leave headroom.
• Forgetting other clients of the same database. Migrations, cron jobs, a second Hyperdrive config and admin tools all share the database’s connection limit.
• Caching a client in a module variable “for speed”. It saves nothing, because Hyperdrive already removes the setup cost, and it causes every error in the stale client section.
• Load testing with local wrangler dev. It bypasses the pool, so the test proves nothing about pool exhaustion.
• Only watching errors. By the time errors show, clients have already queued. Waiting clients show the problem earlier.

Alerts worth setting on a Hyperdrive config

Pull the two datasets on a schedule into whatever tool pages your team, and alert on:

1. Peak waiting clients above zero for several minutes in hyperdrivePoolSizesAdaptiveGroups. This is the earliest sign of contention.
2. Peak open connections close to maxPoolSize for a sustained period. Time to find the slow transaction or ask for more connections.
3. Error share by eventStatus in hyperdriveQueriesAdaptiveGroups, compared with the previous hour.
4. Average connectionLatency rising. Slow new connections to the origin often come before refusals.

I built and led the Workers observability team at Cloudflare, and my advice is the same for every pooled resource. Put the pool chart and your deploy markers on one timeline. A config change or deploy that shrinks the pool, or lengthens a transaction, is much easier to spot when the waiting clients line starts climbing right after it.

Tracing a Cloudflare alert back to its change with Polylane

Polylane connects Cloudflare as a cloud account whose logs and metrics it can query. It triages the account’s alerts and runs scheduled checks against its resources. When a check or alert fires, a fix run works out the cause, citing the evidence for each step, and opens the fix as a pull request you review.

Running on Cloudflare? See how Polylane monitors Cloudflare in production.