What are the advantages/disadvantages of starting a new `ThreadPoolExecutor` when doing `loop.run_in_executor`?

12 hours ago 2
ARTICLE AD BOX

Looking at the documentation for Event Loop, there is this example:

import asyncio import concurrent.futures def blocking_io(): # File operations (such as logging) can block the # event loop: run them in a thread pool. with open('/dev/urandom', 'rb') as f: return f.read(100) def cpu_bound(): # CPU-bound operations will block the event loop: # in general it is preferable to run them in a # process pool. return sum(i * i for i in range(10 ** 7)) async def main(): loop = asyncio.get_running_loop() ## Options: # 1. Run in the default loop's executor: result = await loop.run_in_executor( None, blocking_io) print('default thread pool', result) # 2. Run in a custom thread pool: with concurrent.futures.ThreadPoolExecutor() as pool: result = await loop.run_in_executor( pool, blocking_io) print('custom thread pool', result) # 3. Run in a custom process pool: with concurrent.futures.ProcessPoolExecutor() as pool: result = await loop.run_in_executor( pool, cpu_bound) print('custom process pool', result) # 4. Run in a custom interpreter pool: with concurrent.futures.InterpreterPoolExecutor() as pool: result = await loop.run_in_executor( pool, cpu_bound) print('custom interpreter pool', result) if __name__ == '__main__': asyncio.run(main())

Assuming an IO-bound task, what are the advantages of doing option 2 as opposed to the default ThreadPoolExecutor? Is it just that I can control how it's initialized (num_workers, initializer, etc.), or is there something else under the hood?

Read Entire Article