Initialize interpeter before it runs main script

I need to run python script with costly initialization (0.5 seconds). To mitigate initialization time multiple interpreters are pooled, so when need to execute arises, interpreter is taken from pool and immediately runs given script

def createInterpreters(): for i in range(coreCount): createInterpreter().whenInitializationComplete(addToPool) def runScript(code): interp = getFromPool() interp.runScript(code)

This approach works well in general case but there's some problems when script touches UI (pyqt, tk) because actual interpreter is C# process that dynamically loads python using pythonnet, does costly initialization and reports back to user-facing app. I want to rewrite how interpreters are created: start python process that will preload C# infrastructure and wait for user script.

From python's standpoint it should look like

initCsharp() path = waitForScriptPath() runScript(path)

"Advanced search engines" recommend redirect stdin/out/err, write each line, flush, read output

proc = subprocess.Popen( ['python3'], # Use 'python' if python3 isn't needed stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, # Use text mode for easier string handling (Python 3.7+) bufsize=0 # Unbuffered for immediate interaction ) init_script = "open('asd.txt', 'wb')" # perform side-effect to see whether it's working proc.stdin.write(init_script + '\n') proc.stdin.flush() # file isn't created here, even if I wait for couple of seconds user_script = "print(1)" proc.stdin.write(user_script + '\n') proc.stdin.flush() # Critical: ensures line is sent immediately proc.stdin.close() proc.wait()

The problem with this approach is that script does not run until stdin is closed, so initialization is performed when user launches their script.

Now solution that ~almost~ works: it uses PYTHONSTARTUP env variable to preconfigure python and launches user's script using interactive mode

# startup.py import sys sys.ps1 = "" sys.ps2 = "" sys.__interactivehook__ = lambda : () del sys proc = subprocess.Popen( ['python3', '-q', '-i'], stdin=subprocess.PIPE, text=True, bufsize=0, env={ "PYTHONSTARTUP": "workdir/startup.py"}) code = """ print(1) print(__name__) raise Exception() """.splitlines() for line in code: proc.stdin.write(line + '\n') proc.stdin.close() proc.wait()

It works, but when exception is raised line is always equal to 1.

So the question is: how to split initialization from actual script run with line information preserved?