ARTICLE AD BOX
Multiprocessing Pool forks workers one-by-one, but task dispatch does not begin until the pool infrastructure (i.e. internal queues, handlers etc.) is fully initialized. This behavior comes from Pool implementation rather than Linux itself.
The possible workaround is to launch Process workers manually instead of using Pool, so each one begins executing immediately after calling start().
1 Comment
What about the initializer? Would that execute in every worker right away, or also wait for all workers to start first?
2026-05-12T00:37:45.537Z+00:00
You haven't enlightened us with your Python code. You can arrange for processes to start (almost) immediately in various ways. This is probably the simplest...
from concurrent.futures import ProcessPoolExecutor from time import sleep, monotonic DELAY = 2 LOOP = 5 def process(message: str) -> None: return print(message) if __name__ == "__main__": with ProcessPoolExecutor() as ppe: for _ in range(LOOP): start = monotonic() ppe.submit(process, "Running") sleep(DELAY) duration = monotonic() - start print(f"{duration=:.4f}s")If you run this, you will note that "Running" appears at ~2 second intervals thereby demonstrating the virtual immediacy of the submission of the task to the process pool.
29.2k3 gold badges11 silver badges28 bronze badges
Explore related questions
See similar questions with these tags.
