How to debug Modal function timeouts

When a Modal Function runs longer than it’s allowed, Modal stops it. Once any retries are used up, the caller gets modal.exception.FunctionTimeoutError. Three other limits look like the same failure: the container startup timeout, the 150 second limit on web requests, and the heartbeat timeout.

Debugging comes down to two things. Find out which of these four you hit, then find the step inside your code where the time went.

What a Modal function timeout means

A Modal function timeout means one execution attempt ran longer than the Function’s timeout setting, so Modal stopped it. The default is 300 seconds, and you can set anything from 1 second to 24 hours. The clock only counts time your code is executing. Time spent waiting to be scheduled doesn’t count. The timeout is a cap on one attempt’s runtime. It is the symptom, and the cause is whatever your code was doing when the clock ran out: waiting on a network call, spinning in a loop, loading a model, or holding the GIL.

The four Modal timeouts that look like one

Execution timeout

This is the timeout argument on @app.function or @app.cls. It applies to each execution attempt. Retries start a fresh clock, so a Function with retries=3 can use its full timeout four times. Modal’s docs give the example: an infinite loop with a 100 second timeout and 3 retries runs for at least 400 seconds. The timeout is also loose at the edges. A Function runs for at least as long as its timeout, and it may run a handful of seconds longer.

Container startup timeout

startup_timeout covers the time a container takes to start. That includes imports in global scope and any @modal.enter methods, which is where large model loads usually sit. It was added in Modal v1.1.4. On older clients, or when you leave it unset, timeout covers startup and execution together. A slow model load can then eat the budget before your function body even runs.

Web request timeout

Every web Function type (modal.fastapi_endpoint, modal.asgi_app, modal.wsgi_app and modal.web_server) has a maximum HTTP request time of 150 seconds. The Function behind it can have a longer timeout. When it takes longer than 150 seconds, Modal returns a 303 redirect to a result URL. Browsers follow up to 20 of these redirects, which allows about 50 minutes in total. Clients that don’t follow redirects see the 303 as a failure.

Heartbeat timeout

The Modal client inside each container runs a heartbeat loop. The host uses it to check that the main process is healthy. If heartbeats stop for minutes, Modal kills the container and writes a heartbeat timeout to the logs. Modal names two usual causes: code that holds Python’s Global Interpreter Lock (GIL) for a long time, so the heartbeat thread can’t run, and a process that starts shutting down but never finishes.

graph TB A[A Modal call fails or hangs] --> B{Logs say heartbeat timeout?} B -->|yes| C[GIL held or shutdown stuck] B -->|no| D{Caller got a 303 redirect?} D -->|yes| E[Web request passed 150 seconds] D -->|no| F{First call on a new container?} F -->|yes| G[Startup timeout: imports or enter] F -->|no| H[Execution timeout: find the slow step] style C fill:#fee2e2,stroke:#fca5a5,color:#7f1d1d style E fill:#fee2e2,stroke:#fca5a5,color:#7f1d1d style G fill:#fee2e2,stroke:#fca5a5,color:#7f1d1d style H fill:#fee2e2,stroke:#fca5a5,color:#7f1d1d

Finding which timeout hit you, step by step

1. Pull the logs around the failure. modal app logs fetches only the last 100 entries by default, so give it a time range and tag each line with the call and container IDs:

modal app logs my-app --since 2h --search timeout --timestamps --show-function-call-id --show-container-id

2. Read Modal’s own system logs. Heartbeat timeouts come from Modal itself, and Modal’s system-level logs may show them:

modal app logs my-app --since 2h --source system --timestamps --show-container-id

3. Follow one call from start to end. Take the fc- ID from step 1 and fetch only that call’s lines. The last line before the silence tells you where it stopped:

modal app logs my-app --function-call fc-abc123 --timestamps

4. Check the App page. Every App logs a dashboard link when it starts, and modal app dashboard my-app opens it. It shows function call history with success and failure counts, plus CPU, RAM and GPU metrics. A sudden run of failures right after a deploy points to a code change. Use modal app history my-app to see when deploys happened.

5. Rule startup in or out. If only the first call on each new container times out, the time is going into global scope imports or @modal.enter. If inputs spend too much time in the pending state, Modal’s cold start guide says to target queueing time.

6. Look inside a container that is stuck right now. Open your Function’s dashboard, go to the Containers tab and use Live Profiling to see what code is running. From the terminal, list the containers and attach a debug shell, which comes with py-spy, strace and ps:

modal container list
modal shell ta-01JK47GVDMWMGPH8MQ0EW30Y25

Inside the shell, find your Python process and dump its stack:

ps aux | grep python
py-spy dump --pid 1

Replace 1 with the PID from ps. A stack parked in a socket read is waiting on an upstream service. A stack deep in your own loop is a loop. The shell closes as soon as the container finishes, so do this while the call is still hanging.

7. Turn on Modal client debug logs when the trouble looks like a shutdown or heartbeat problem. Set MODAL_LOGLEVEL through a Secret so it applies inside the container:

@app.function(secrets=[modal.Secret.from_dict({"MODAL_LOGLEVEL": "DEBUG"})])
def f():
    ...

Timing every step inside the function

A FunctionTimeoutError tells you the attempt ran out of time. It doesn’t tell you which call it was waiting on. I built and led the Workers observability team at Cloudflare, and the same habit works on Modal: log how long each outbound call and each heavy step takes. Then the last timing line before the silence names the culprit.

import time
import modal

app = modal.App("reports")

@app.function(timeout=600)
def build_report(user_id: str):
    t = time.monotonic()
    rows = fetch_rows(user_id)
    print(f"step=fetch_rows took={time.monotonic() - t:.2f}s")

    t = time.monotonic()
    summary = call_model(rows)
    print(f"step=call_model took={time.monotonic() - t:.2f}s")
    return summary

A healthy call prints both lines. A call that times out prints step=fetch_rows and nothing after it, so the model call never returned. Search for the step names with modal app logs my-app --search step=.

Fixing each cause of a Modal timeout

An upstream call that hangs needs its own deadline

The most common hang is a network call with no timeout of its own. Give every outbound request a deadline well under the Function’s timeout, and turn a failure into an exception you can see:

import httpx

def fetch_rows(user_id: str):
    resp = httpx.get(
        f"https://api.example.com/rows/{user_id}",
        timeout=httpx.Timeout(10.0, connect=5.0),
    )
    resp.raise_for_status()
    return resp.json()

Modal’s own timeout can overrun by a few seconds, and its docs recommend writing your own timeout logic when you need exact limits. For async code, wrap the slow part in asyncio.wait_for:

import asyncio

@app.function(timeout=300)
async def summarise(doc: str):
    try:
        return await asyncio.wait_for(call_model_async(doc), timeout=240)
    except asyncio.TimeoutError:
        print("step=call_model timed out after 240s")
        raise

Set database driver timeouts the same way, for both connecting and querying.

A loop that never exits, multiplied by retries

Look for while loops that wait on data that may never arrive, pagination that follows a cursor without checking whether it changed, and retry loops with no cap. Give each one a hard limit and log the iteration count. Be careful with retries: each attempt gets a full new timeout, so a stuck loop with retries=3 burns four timeouts’ worth of compute before the caller hears about it.

Long work that needs more time

If the work is long by nature, like a batch job, a training step or a long inference, raise timeout. It goes up to 24 hours:

@app.function(timeout=600)
def nightly_import():
    ...

Work made of many independent parts often finishes sooner when you split it with Function.map, so inputs run in parallel, each with its own timeout. With map, each input is also retried on its own.

Slow startup: model loads and heavy imports

When the first call on each container is the one that fails, give startup its own budget and keep timeout for the real work. This needs Modal v1.1.4 or later:

@app.cls(startup_timeout=600, timeout=120, gpu="L4")
class Model:
    @modal.enter()
    def load(self):
        self.model = load_weights("/models/my-model")

    @modal.method()
    def predict(self, x):
        return self.model(x)

Then make startup shorter. Download model weights ahead of time into the Image or a Volume, so they aren’t fetched on every boot. Modal’s docs say that for models in the tens of gigabytes, this can bring boot times down from minutes to seconds. Load independent files at the same time from a thread pool. Memory Snapshots can cut the cost of imports across cold starts. If a startup delay still can’t be avoided, keep warm containers with min_containers, buffer_containers or a longer scaledown_window, and accept that you pay for them while they sit idle.

A web endpoint that runs past 150 seconds

If a request regularly takes longer than 150 seconds, stop making the caller wait on it. Have the endpoint start the work with .spawn(), return the call ID, and let the client poll a second endpoint. This is the pattern Modal documents:

import fastapi
import modal

image = modal.Image.debian_slim().pip_install("fastapi[standard]")
app = modal.App(image=image)
web_app = fastapi.FastAPI()

@app.function()
@modal.asgi_app()
def fastapi_app():
    return web_app

@app.function(timeout=600)
def slow_operation():
    ...

@web_app.post("/accept")
async def accept_job(request: fastapi.Request):
    call = slow_operation.spawn()
    return {"call_id": call.object_id}

@web_app.get("/result/{call_id}")
async def poll_results(call_id: str):
    function_call = modal.FunctionCall.from_id(call_id)
    try:
        return function_call.get(timeout=0)
    except TimeoutError:
        return fastapi.responses.JSONResponse({}, status_code=202)

A TimeoutError from get(timeout=0) here just means the result isn’t ready yet, and the endpoint answers 202. Only treat FunctionTimeoutError as a real failure. If you keep the long request and depend on redirects, make sure the client follows them: curl -L does.

A heartbeat timeout from a held GIL

Use py-spy in a debug shell to confirm that a thread is holding the GIL. Modal’s suggested quick fix is to move the code that holds the GIL into a subprocess, so the heartbeat thread in the main process keeps running:

import subprocess

@app.function(timeout=900)
def crunch(path: str):
    subprocess.run(["python", "/root/crunch.py", path], check=True)

If the logs show a shutdown that starts and never finishes, look for cleanup code that blocks, like a thread that is never joined or a connection close that waits forever.

Handle the timeout where you call the Function

Catch the error at the call site, so a timeout becomes a clear result and the caller doesn’t crash:

import modal.exception

@app.local_entrypoint()
def main():
    try:
        build_report.remote("user_123")
    except modal.exception.FunctionTimeoutError:
        print("build_report timed out; check logs for the last step= line")

Where Modal timeout fixes go wrong

• Raising timeout over a hang. A call that never returns uses a longer timeout in full, just as it used 300 seconds, and with retries it uses it more than once. Fix: add the deadline on the upstream call first. Raise the timeout only for work that finishes.
• Old clients where timeout covers startup. Before v1.1.4, or with startup_timeout unset, model loading counts against timeout. Fix: upgrade and set startup_timeout on its own.
• Python’s urllib stopping redirects early. As of March 2025, urllib allows at most 4 redirects to one URL by default. That caps a web Function at 750 seconds (5 × 150 s) unless you change the setting. Fix: raise the limit, or use the spawn and poll pattern.
• Long requests that need CORS. The 303 redirect trick doesn’t work with CORS requests, because your code hasn’t returned yet when the server has to add CORS headers. Fix: spawn and poll.
• Expecting exact cut-offs. Modal can let a Function run a few seconds past its timeout. Fix: put the precise limit in your own code with asyncio.wait_for or client timeouts.
• Reading only the last 100 log lines. modal app logs stops at 100 entries unless you pass --tail or --since. Fix: always give it a time window, and filter by --function-call.
• Calling Modal from forked workers. A child process from Celery prefork or multiprocessing can inherit stale network state from its parent and run into connection problems. Fix: create a fresh modal.Client.from_credentials after the fork and pass it in.
• Debugging a deploy that still runs old code. With a rolling deploy, inputs can land on containers from the previous version. Fix: deploy with --strategy=recreate while you test, or roll back with modal app rollback my-app if a deploy caused the timeouts.

Keeping Modal timeouts from coming back

Set a deadline on every outbound call, keep the step= timing lines on the calls that matter, and catch FunctionTimeoutError at every call site so a timeout always leaves a log line. Set timeout and startup_timeout on purpose for each Function. A value near the real runtime catches a hang early, while one far above it hides a hang behind a big compute bill. Check function call history on the App page after each deploy. If you send Modal telemetry to Datadog, alert on timeouts per Function there.

Catching Modal timeouts with Polylane

Polylane detects production issues and fixes them by opening pull requests. It connects to Modal, so when a Modal app starts failing it works out why. When the cause is in your code, the fix arrives as a pull request on the repo behind the app, ready for your team’s normal review.

Running on Modal? See how Polylane monitors Modal in production.