2021-11-12

WARNING: This article may be obsolete
This post was published in 2021-11-12. Obviously, expired content is less useful to users if it has already pasted its expiration date.

This article is categorized as "Garbage" . It should NEVER be appeared in your search engine's results.

python FastAPI与ThreadPoolExecutor/ProcessPoolExecutor

（FastAPI提供的BackgroundTasks可能更为普遍/更高性能）

假设现在有提供rest API服务的FastAPI代码如下：

from fastapi import FastAPI, Request
from concurrent.futures import ThreadPoolExecutor, ProcessPoolExecutor
import time

executor = ThreadPoolExecutor(6)

app = FastAPI()


def send_data():
    time.sleep(1)
    gmt_time = time.gmtime(time.time())
    print(time.strftime('%Y-%m-%d %H:%M:%S', gmt_time))
    
    
    
@app.get("/test")
def get_body(request: Request):
    executor.submit(send_data)
    return "ok"

假设它在本机23333端口运行：

$  uvicorn main:app --reload --host 127.0.0.1 --port 23333

另有一个stress.py代码，用线程池向FastAPI服务请求 /test 接口：

from concurrent.futures import ThreadPoolExecutor, ProcessPoolExecutor
import requests


def download(url):
    print(requests.get(url).text)


if __name__ == '__main__':
    urlList = ['http://127.0.0.1:23333/test'] * 30

    pool = ThreadPoolExecutor(max_workers=30)
    pool.map(download, urlList)

注意FastAPI代码的第5行，当前使用的是 ThreadPoolExecutor 。

运行结果如下：

这段程序会消耗数秒钟来（依次）打印时间、暂停、打印时间。在这个过程中，即使中途 uvicorn 热更新强制重启，这个步骤仍然不会中断，直到所有线程池里的任务执行完毕为止。

ThreadPoolExecutor和ProcessPoolExecutor在当前代码里差异不明显，ProcessPoolExecutor在打印时间的时候可能会出现轻微的“不完全协调”，即处于同一个ProcessPool执行的时间打印能用肉眼区分出执行的先后顺序。