Python 3.14t free-thread compatibility with Polars

11 hours ago 2
ARTICLE AD BOX

I started using Python3.14t (free-threaded build) recently and had some blast. However, when I install Polars

python3.14t -m pip install polars

The process stuck at the last line of the following

Collecting polars Using cached polars-1.35.2-py3-none-any.whl.metadata (10 kB) Collecting polars-runtime-32==1.35.2 (from polars) Using cached polars_runtime_32-1.35.2.tar.gz (2.7 MB) Installing build dependencies ... done Getting requirements to build wheel ... done Installing backend dependencies ... done Preparing metadata (pyproject.toml) ... done Using cached polars-1.35.2-py3-none-any.whl (783 kB) Building wheels for collected packages: polars-runtime-32 Building wheel for polars-runtime-32 (pyproject.toml) ...

for a long time (5 - 6 minutes) with all my CPUs firing up. But eventually it got through.

Now when I run

python3.14t

and import the library

import polars

The console prompted:

<frozen importlib._bootstrap>:491: RuntimeWarning: The global interpreter lock (GIL) has been enabled to load module '_polars_runtime_32._polars_runtime_32', which has not declared that it can run safely without the GIL. To override this behavior and keep the GIL disabled (at your own risk), run with PYTHON_GIL=0 or -Xgil=0.

So I rerun Python as:

python3.14t -Xgil=0

I know Polars operations are mostly multithreaded, so if we use it plainly in a multithreaded environment, resource contention might be insane which I speculate is why the installation took so long and used all my CPU cores.

However, in some scenarios I do want Polars to work on many pieces of different data in parallel. In these scenarios, if I preset

os.environ["POLARS_MAX_THREADS"] = "1"

before parallelization (by using a thread pool like concurrent.futures.ThreadPoolExecutor), will there be any pitfalls? And more importantly, is

python3.14t -Xgil=0

even safe at all for using Polars?

Thank you!

Read Entire Article