Why Does FastAPI Execute API Calls Serially Instead of in Parallel?

FastAPI: Why API Calls Are Executed Serially Instead of in Parallel

FastAPI defines endpoints (also known as path operation functions) using both async def and def. While conceptually async def may suggest parallelization, FastAPI actually handles these functions differently:

Endpoints Defined with async def:

• Run directly in the event loop.
• Can only be called from other async functions and must pause (await) before executing non-async operations, such as I/O.
• Ensure the event loop is not blocked and that other tasks can be executed concurrently.

Endpoints Defined with def:

• Run not directly in the event loop but instead in a separate thread from an external threadpool.
• Can be called from either async or non-async functions.
• May block the event loop and prevent other tasks from executing if non-async operations are performed without pausing.
• Offer performance optimizations in certain scenarios.

Impact on Parallelization:

Based on this understanding, let's examine your code example:

@app.get("/ping")
async def ping(request: Request):
    print("Hello")
    time.sleep(5) # This sleeps the event loop for 5 seconds
    print("bye")
    return {"ping": "pong!"}

In this case, the following occurs:

• Two requests to /ping are sent simultaneously.
• The async defENDPOINT runs directly in the event loop.
• The time.sleep(5) call pauses the event loop for 5 seconds.
• During these 5 seconds, the second request is queued and cannot be processed since the event loop is blocked.
• Once the event loop resumes after 5 seconds, the second request is processed.

As a result, the responses are printed serially:

Hello
bye
Hello
bye

To enable parallelization, non-async operations like time.sleep() should not be used in async def endpoints. Instead, one of the following approaches can be applied:

• Use run_in_threadpool() to spawn a thread and execute the blocking operation outside the event loop.
• Use loop.run_in_executor() or asyncio.to_thread() to run the blocking operation in a separate thread or process.
• Consider using ThreadPoolExecutor or ProcessPoolExecutor to run computationally intensive tasks off-process.

The above is the detailed content of Why Does FastAPI Execute API Calls Serially Instead of in Parallel?. For more information, please follow other related articles on the PHP Chinese website!