EC2ThrottledException: fix Lambda EC2 API throttling

EC2ThrottledException means Amazon EC2 throttled Lambda during function initialization using the execution role supplied for the function. The Lambda Invoke reference assigns it HTTP 502. Confirm that response, inspect VPC preparation and EC2 API activity in the same account and Region, then retry only after addressing the request pressure.

Raising Lambda reserved concurrency does not raise an EC2 API request limit. Spreading out network configuration work or reducing a competing EC2 poller can help, but first identify which API and lifecycle operation failed.

Meaning: distinguish three different failures

These names come from the Invoke error table. A handler’s own EC2 SDK call can instead fail with EC2’s RequestLimitExceeded. That is application execution, not necessarily the Lambda service’s initialization exception. Preserve the service name, operation and error code in caller logs so those paths stay distinguishable.

Lambda’s metric definitions say rejected invocation requests do not count as Invocations or function Errors. An empty handler log is therefore compatible with a pre-execution failure. Do not use a quiet error chart as proof that every invocation request worked.

Modern Hyperplane ENIs change the diagnosis

The current VPC guide describes shared Hyperplane ENIs, associated with subnet and security-group combinations. Functions in the same account can reuse those interfaces when their combinations match. Each interface supports up to 65,000 connections or ports, and Lambda adds interfaces when network traffic and concurrency require them.

This is not the historical model of creating an interface for every execution environment. A traffic burst does not imply one new ENI per invocation. New network combinations, lifecycle recreation and competing requests to the same EC2 API are hypotheses to verify rather than inevitable consequences of invocation volume.

For a new function, interface preparation can take several minutes and the function stays Pending until ready. For an existing function, additional operations such as code updates or version creation can be blocked while a new interface is prepared, but previous versions can continue serving invocations. An update problem is therefore not equivalent to an outage of every deployed version.

AWS also documents reclamation of unused interfaces after 14 idle days. An affected function becomes Inactive; its next invocation attempt fails and preparation restarts. Read the returned error and state reason before calling that lifecycle an EC2 throttling incident. Reclamation alone does not prove a rate bucket was exhausted.

Confirm the operation, target and state

Use AWS CLI v2 with the affected account and explicit Region:

export AWS_PROFILE=production-reader
export AWS_REGION=us-east-1
FUNCTION=orders-worker
aws sts get-caller-identity
aws lambda get-function-configuration \
  --function-name "$FUNCTION" --region "$AWS_REGION" \
  --query '{State:State,Reason:StateReason,ReasonCode:StateReasonCode,Update:LastUpdateStatus,UpdateReason:LastUpdateStatusReason,UpdateCode:LastUpdateStatusReasonCode,VPC:VpcConfig,Role:Role}'

In Lambda → Functions → your function → Configuration → VPC, inspect subnets and security groups. If the caller invokes an alias, inspect the alias and its published version too. A healthy $LATEST does not prove an alias points to it.

Save the failed request’s UTC timestamp, Region, function ARN, qualifier, request ID and exact error from the caller’s SDK or CLI output. Separate UpdateFunctionConfiguration failures from Invoke failures: state reason codes describe preparation and update work, while an Invoke exception describes an invocation request. These observations can correlate without being interchangeable.

For reproduction, use a harmless event or a dedicated test function. Invoke executes real code; replaying a payment event merely to capture a response can repeat side effects. A read-only configuration check is a better first operation.

Find competing EC2 API requests

EC2 throttles by account, Region and API action. Token buckets provide burst capacity and refill at a sustained rate. Some actions have their own buckets, and some unfiltered, unpaginated describe requests use smaller buckets. A RunInstances limit is not a DescribeNetworkInterfaces limit.

Open CloudTrail → Event history in the incident Region, filter Event source to ec2.amazonaws.com, and inspect recorded operations in the failure window. The CLI starting point is:

aws cloudtrail lookup-events --region "$AWS_REGION" \
  --lookup-attributes AttributeKey=EventSource,AttributeValue=ec2.amazonaws.com \
  --start-time 2026-10-06T10:00:00Z \
  --end-time 2026-10-06T11:00:00Z \
  --query 'Events[].{Time:EventTime,Name:EventName,Event:CloudTrailEvent}'

Replace those times with the incident. Inspect caller identity, parameters and errors for network-interface activity. Event history is not a complete request-rate counter; an absent event does not rule out throttling. Correlate caller logs and the EC2 API usage/throttle monitoring described in the guide.

In Service Quotas → Amazon EC2, search the implicated action’s request bucket maximum capacity and refill rate. Read applied quotas rather than assuming published defaults equal your account’s effective limit. Gather available EC2 request IDs and the workload responsible before requesting an increase.

Fix the evidence-backed cause

Reduce bursts and polling. Stagger infrastructure jobs that target the same APIs and pause unnecessary inventory tasks. Filter and paginate describe requests. A tight readiness loop can add API pressure while the resource is still preparing; use bounded polling with an increasing interval.

Reuse compatible network configurations. AWS recommends the same subnet/security-group combinations where appropriate so Lambda can reuse Hyperplane ENIs. Keep required network isolation: merging security groups solely to reduce interface creation changes access and requires an independent design review.

Wait for an update before issuing the next one. After configuration changes, use the CLI waiter instead of immediately publishing or updating again:

aws lambda wait function-updated-v2 \
  --function-name "$FUNCTION" --region "$AWS_REGION"
aws lambda get-function-configuration \
  --function-name "$FUNCTION" --region "$AWS_REGION" \
  --query '[State,LastUpdateStatus,StateReasonCode,LastUpdateStatusReasonCode]'

A waiter failure should lead back to the state reasons, not an infinite retry loop. New-function activation and existing-function update completion are related but different checks.

Retry transient requests with limits. EC2 recommends exponential backoff, jitter, a maximum delay and maximum attempts. Use SDK retry configuration instead of layering an unbounded application loop over automatic retries. For AWS CLI:

export AWS_RETRY_MODE=standard
export AWS_MAX_ATTEMPTS=5

Those settings enable the CLI retry policy; they do not guarantee every error is retried or that capacity recovers. Keep an overall deadline and preserve the final error. For side-effecting invocations, use an application idempotency key before resending.

Request the specific rate increase when needed. Follow the EC2 guide’s support process with the account, Region, action and incident evidence. An ENI-count quota increase does not increase an API bucket, and Lambda concurrency does not either. Access-denied failures require permissions work rather than rate-limit work.

Verification and prevention

Confirm State=Active and LastUpdateStatus=Successful, then send a safe event through the actual production alias or version. Verify the service response and expected business result. Track caller-side initialization errors alongside native Lambda metrics so rejected requests remain visible.

For asynchronous producers, HTTP 202 means queued, not completed. Check AsyncEventsDropped and event age for backlog or discarded work. For system-level evidence once code runs, use the LambdaInsightsExtension checks.

Record EC2 callers, polling frequencies and network combinations created by infrastructure jobs. Alert on the specific failed API and inspect state reasons before changing capacity. An automated investigator should receive the read permissions needed for evidence separately from identities that change networking.

Polylane and Lambda start-up failures

Polylane’s AWS integration connects it to AWS, so it can see when a function fails to start and work out why. The AWS overview has more. EC2 request quotas and retry behaviour still require the corresponding AWS and application fixes.

Running on AWS? See how Polylane monitors AWS in production.