ARTICLE AD BOX
I am trying to load Qwen on google colab.
cell 1:
cell 1 results:
Requirement already satisfied: transformers>=4.44.0 in /usr/local/lib/python3.12/dist-packages (5.0.0) Collecting transformers>=4.44.0 Downloading transformers-5.3.0-py3-none-any.whl.metadata (32 kB) Requirement already satisfied: accelerate>=0.32.0 in /usr/local/lib/python3.12/dist-packages (1.13.0) Collecting bitsandbytes>=0.49.1 Downloading bitsandbytes-0.49.2-py3-none-manylinux_2_24_x86_64.whl.metadata (10 kB) Requirement already satisfied: torch>=2.3.0 in /usr/local/lib/python3.12/dist-packages (2.10.0+cu128) Collecting torch>=2.3.0 Downloading torch-2.11.0-cp312-cp312-manylinux_2_28_x86_64.whl.metadata (29 kB) Requirement already satisfied: safetensors>=0.4.0 in /usr/local/lib/python3.12/dist-packages (0.7.0) Requirement already satisfied: torchvision in /usr/local/lib/python3.12/dist-packages (0.25.0+cu128) Collecting torchvision Downloading torchvision-0.26.0-cp312-cp312-manylinux_2_28_x86_64.whl.metadata (5.5 kB) Requirement already satisfied: torchaudio in /usr/local/lib/python3.12/dist-packages (2.10.0+cu128) Collecting torchaudio Downloading torchaudio-2.11.0-cp312-cp312-manylinux_2_28_x86_64.whl.metadata (6.9 kB) Requirement already satisfied: huggingface-hub<2.0,>=1.3.0 in /usr/local/lib/python3.12/dist-packages (from transformers>=4.44.0) (1.7.1) Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.12/dist-packages (from transformers>=4.44.0) (2.0.2) Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.12/dist-packages (from transformers>=4.44.0) (26.0) Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.12/dist-packages (from transformers>=4.44.0) (6.0.3) Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.12/dist-packages (from transformers>=4.44.0) (2025.11.3) Requirement already satisfied: tokenizers<=0.23.0,>=0.22.0 in /usr/local/lib/python3.12/dist-packages (from transformers>=4.44.0) (0.22.2) Requirement already satisfied: typer in /usr/local/lib/python3.12/dist-packages (from transformers>=4.44.0) (0.24.1) Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.12/dist-packages (from transformers>=4.44.0) (4.67.3) Requirement already satisfied: psutil in /usr/local/lib/python3.12/dist-packages (from accelerate>=0.32.0) (5.9.5) Requirement already satisfied: filelock in /usr/local/lib/python3.12/dist-packages (from torch>=2.3.0) (3.25.2) Requirement already satisfied: typing-extensions>=4.10.0 in /usr/local/lib/python3.12/dist-packages (from torch>=2.3.0) (4.15.0) Requirement already satisfied: setuptools<82 in /usr/local/lib/python3.12/dist-packages (from torch>=2.3.0) (75.2.0) Requirement already satisfied: sympy>=1.13.3 in /usr/local/lib/python3.12/dist-packages (from torch>=2.3.0) (1.14.0) Requirement already satisfied: networkx>=2.5.1 in /usr/local/lib/python3.12/dist-packages (from torch>=2.3.0) (3.6.1) Requirement already satisfied: jinja2 in /usr/local/lib/python3.12/dist-packages (from torch>=2.3.0) (3.1.6) Requirement already satisfied: fsspec>=0.8.5 in /usr/local/lib/python3.12/dist-packages (from torch>=2.3.0) (2025.3.0) Collecting cuda-toolkit==13.0.2 (from cuda-toolkit[cublas,cudart,cufft,cufile,cupti,curand,cusolver,cusparse,nvjitlink,nvrtc,nvtx]==13.0.2; platform_system == "Linux"->torch>=2.3.0) Downloading cuda_toolkit-13.0.2-py2.py3-none-any.whl.metadata (9.4 kB) Collecting cuda-bindings<14,>=13.0.3 (from torch>=2.3.0) Downloading cuda_bindings-13.2.0-cp312-cp312-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata (2.3 kB) Collecting nvidia-cudnn-cu13==9.19.0.56 (from torch>=2.3.0) Downloading nvidia_cudnn_cu13-9.19.0.56-py3-none-manylinux_2_27_x86_64.whl.metadata (1.9 kB) Collecting nvidia-cusparselt-cu13==0.8.0 (from torch>=2.3.0) Downloading nvidia_cusparselt_cu13-0.8.0-py3-none-manylinux2014_x86_64.whl.metadata (12 kB) Collecting nvidia-nccl-cu13==2.28.9 (from torch>=2.3.0) Downloading nvidia_nccl_cu13-2.28.9-py3-none-manylinux_2_18_x86_64.whl.metadata (2.0 kB) Collecting nvidia-nvshmem-cu13==3.4.5 (from torch>=2.3.0) Downloading nvidia_nvshmem_cu13-3.4.5-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (2.1 kB) Requirement already satisfied: triton==3.6.0 in /usr/local/lib/python3.12/dist-packages (from torch>=2.3.0) (3.6.0) Collecting nvidia-cublas==13.1.0.3.* (from cuda-toolkit[cublas,cudart,cufft,cufile,cupti,curand,cusolver,cusparse,nvjitlink,nvrtc,nvtx]==13.0.2; platform_system == "Linux"->torch>=2.3.0) Downloading nvidia_cublas-13.1.0.3-py3-none-manylinux_2_27_x86_64.whl.metadata (1.7 kB) Collecting nvidia-cuda-runtime==13.0.96.* (from cuda-toolkit[cublas,cudart,cufft,cufile,cupti,curand,cusolver,cusparse,nvjitlink,nvrtc,nvtx]==13.0.2; platform_system == "Linux"->torch>=2.3.0) Downloading nvidia_cuda_runtime-13.0.96-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.7 kB) Collecting nvidia-cufft==12.0.0.61.* (from cuda-toolkit[cublas,cudart,cufft,cufile,cupti,curand,cusolver,cusparse,nvjitlink,nvrtc,nvtx]==13.0.2; platform_system == "Linux"->torch>=2.3.0) Downloading nvidia_cufft-12.0.0.61-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.8 kB) Collecting nvidia-cufile==1.15.1.6.* (from cuda-toolkit[cublas,cudart,cufft,cufile,cupti,curand,cusolver,cusparse,nvjitlink,nvrtc,nvtx]==13.0.2; platform_system == "Linux"->torch>=2.3.0) Downloading nvidia_cufile-1.15.1.6-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.7 kB) Collecting nvidia-cuda-cupti==13.0.85.* (from cuda-toolkit[cublas,cudart,cufft,cufile,cupti,curand,cusolver,cusparse,nvjitlink,nvrtc,nvtx]==13.0.2; platform_system == "Linux"->torch>=2.3.0) Downloading nvidia_cuda_cupti-13.0.85-py3-none-manylinux_2_25_x86_64.whl.metadata (1.7 kB) Collecting nvidia-curand==10.4.0.35.* (from cuda-toolkit[cublas,cudart,cufft,cufile,cupti,curand,cusolver,cusparse,nvjitlink,nvrtc,nvtx]==13.0.2; platform_system == "Linux"->torch>=2.3.0) Downloading nvidia_curand-10.4.0.35-py3-none-manylinux_2_27_x86_64.whl.metadata (1.7 kB) Collecting nvidia-cusolver==12.0.4.66.* (from cuda-toolkit[cublas,cudart,cufft,cufile,cupti,curand,cusolver,cusparse,nvjitlink,nvrtc,nvtx]==13.0.2; platform_system == "Linux"->torch>=2.3.0) Downloading nvidia_cusolver-12.0.4.66-py3-none-manylinux_2_27_x86_64.whl.metadata (1.8 kB) Collecting nvidia-cusparse==12.6.3.3.* (from cuda-toolkit[cublas,cudart,cufft,cufile,cupti,curand,cusolver,cusparse,nvjitlink,nvrtc,nvtx]==13.0.2; platform_system == "Linux"->torch>=2.3.0) Downloading nvidia_cusparse-12.6.3.3-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.8 kB) Collecting nvidia-nvjitlink==13.0.88.* (from cuda-toolkit[cublas,cudart,cufft,cufile,cupti,curand,cusolver,cusparse,nvjitlink,nvrtc,nvtx]==13.0.2; platform_system == "Linux"->torch>=2.3.0) Downloading nvidia_nvjitlink-13.0.88-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl.metadata (1.7 kB) Collecting nvidia-cuda-nvrtc==13.0.88.* (from cuda-toolkit[cublas,cudart,cufft,cufile,cupti,curand,cusolver,cusparse,nvjitlink,nvrtc,nvtx]==13.0.2; platform_system == "Linux"->torch>=2.3.0) Downloading nvidia_cuda_nvrtc-13.0.88-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl.metadata (1.7 kB) Collecting nvidia-nvtx==13.0.85.* (from cuda-toolkit[cublas,cudart,cufft,cufile,cupti,curand,cusolver,cusparse,nvjitlink,nvrtc,nvtx]==13.0.2; platform_system == "Linux"->torch>=2.3.0) Downloading nvidia_nvtx-13.0.85-py3-none-manylinux1_x86_64.manylinux_2_5_x86_64.whl.metadata (1.8 kB) Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /usr/local/lib/python3.12/dist-packages (from torchvision) (11.3.0) Requirement already satisfied: cuda-pathfinder~=1.1 in /usr/local/lib/python3.12/dist-packages (from cuda-bindings<14,>=13.0.3->torch>=2.3.0) (1.4.3) Requirement already satisfied: hf-xet<2.0.0,>=1.4.2 in /usr/local/lib/python3.12/dist-packages (from huggingface-hub<2.0,>=1.3.0->transformers>=4.44.0) (1.4.2) Requirement already satisfied: httpx<1,>=0.23.0 in /usr/local/lib/python3.12/dist-packages (from huggingface-hub<2.0,>=1.3.0->transformers>=4.44.0) (0.28.1) Requirement already satisfied: mpmath<1.4,>=1.1.0 in /usr/local/lib/python3.12/dist-packages (from sympy>=1.13.3->torch>=2.3.0) (1.3.0) Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.12/dist-packages (from jinja2->torch>=2.3.0) (3.0.3) Requirement already satisfied: click>=8.2.1 in /usr/local/lib/python3.12/dist-packages (from typer->transformers>=4.44.0) (8.3.1) Requirement already satisfied: shellingham>=1.3.0 in /usr/local/lib/python3.12/dist-packages (from typer->transformers>=4.44.0) (1.5.4) Requirement already satisfied: rich>=12.3.0 in /usr/local/lib/python3.12/dist-packages (from typer->transformers>=4.44.0) (13.9.4) Requirement already satisfied: annotated-doc>=0.0.2 in /usr/local/lib/python3.12/dist-packages (from typer->transformers>=4.44.0) (0.0.4) Requirement already satisfied: anyio in /usr/local/lib/python3.12/dist-packages (from httpx<1,>=0.23.0->huggingface-hub<2.0,>=1.3.0->transformers>=4.44.0) (4.12.1) Requirement already satisfied: certifi in /usr/local/lib/python3.12/dist-packages (from httpx<1,>=0.23.0->huggingface-hub<2.0,>=1.3.0->transformers>=4.44.0) (2026.2.25) Requirement already satisfied: httpcore==1.* in /usr/local/lib/python3.12/dist-packages (from httpx<1,>=0.23.0->huggingface-hub<2.0,>=1.3.0->transformers>=4.44.0) (1.0.9) Requirement already satisfied: idna in /usr/local/lib/python3.12/dist-packages (from httpx<1,>=0.23.0->huggingface-hub<2.0,>=1.3.0->transformers>=4.44.0) (3.11) Requirement already satisfied: h11>=0.16 in /usr/local/lib/python3.12/dist-packages (from httpcore==1.*->httpx<1,>=0.23.0->huggingface-hub<2.0,>=1.3.0->transformers>=4.44.0) (0.16.0) Requirement already satisfied: markdown-it-py>=2.2.0 in /usr/local/lib/python3.12/dist-packages (from rich>=12.3.0->typer->transformers>=4.44.0) (4.0.0) Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /usr/local/lib/python3.12/dist-packages (from rich>=12.3.0->typer->transformers>=4.44.0) (2.19.2) Requirement already satisfied: mdurl~=0.1 in /usr/local/lib/python3.12/dist-packages (from markdown-it-py>=2.2.0->rich>=12.3.0->typer->transformers>=4.44.0) (0.1.2) Downloading transformers-5.3.0-py3-none-any.whl (10.7 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.7/10.7 MB 93.6 MB/s eta 0:00:00 Downloading bitsandbytes-0.49.2-py3-none-manylinux_2_24_x86_64.whl (60.7 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 60.7/60.7 MB 44.8 MB/s eta 0:00:00 Downloading torch-2.11.0-cp312-cp312-manylinux_2_28_x86_64.whl (530.7 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 530.7/530.7 MB 2.6 MB/s eta 0:00:00 Downloading cuda_toolkit-13.0.2-py2.py3-none-any.whl (2.4 kB) Downloading nvidia_cudnn_cu13-9.19.0.56-py3-none-manylinux_2_27_x86_64.whl (366.1 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 366.1/366.1 MB 3.1 MB/s eta 0:00:00 Downloading nvidia_cusparselt_cu13-0.8.0-py3-none-manylinux2014_x86_64.whl (169.9 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 169.9/169.9 MB 10.0 MB/s eta 0:00:00 Downloading nvidia_nccl_cu13-2.28.9-py3-none-manylinux_2_18_x86_64.whl (196.5 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 196.5/196.5 MB 4.8 MB/s eta 0:00:00 Downloading nvidia_nvshmem_cu13-3.4.5-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (60.4 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 60.4/60.4 MB 43.2 MB/s eta 0:00:00 Downloading nvidia_cublas-13.1.0.3-py3-none-manylinux_2_27_x86_64.whl (423.1 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 423.1/423.1 MB 2.8 MB/s eta 0:00:00 Downloading nvidia_cuda_cupti-13.0.85-py3-none-manylinux_2_25_x86_64.whl (10.7 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.7/10.7 MB 132.0 MB/s eta 0:00:00 Downloading nvidia_cuda_nvrtc-13.0.88-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl (90.2 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 90.2/90.2 MB 28.9 MB/s eta 0:00:00 Downloading nvidia_cuda_runtime-13.0.96-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (2.2 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.2/2.2 MB 90.9 MB/s eta 0:00:00 Downloading nvidia_cufft-12.0.0.61-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (214.1 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 214.1/214.1 MB 4.6 MB/s eta 0:00:00 Downloading nvidia_cufile-1.15.1.6-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (1.2 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 72.8 MB/s eta 0:00:00 Downloading nvidia_curand-10.4.0.35-py3-none-manylinux_2_27_x86_64.whl (59.5 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 59.5/59.5 MB 44.4 MB/s eta 0:00:00 Downloading nvidia_cusolver-12.0.4.66-py3-none-manylinux_2_27_x86_64.whl (200.9 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 200.9/200.9 MB 4.8 MB/s eta 0:00:00 Downloading nvidia_cusparse-12.6.3.3-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (145.9 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 145.9/145.9 MB 17.9 MB/s eta 0:00:00 Downloading nvidia_nvjitlink-13.0.88-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl (40.7 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 40.7/40.7 MB 63.0 MB/s eta 0:00:00 Downloading nvidia_nvtx-13.0.85-py3-none-manylinux1_x86_64.manylinux_2_5_x86_64.whl (148 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 148.0/148.0 kB 14.9 MB/s eta 0:00:00 Downloading torchvision-0.26.0-cp312-cp312-manylinux_2_28_x86_64.whl (7.5 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.5/7.5 MB 104.3 MB/s eta 0:00:00 Downloading torchaudio-2.11.0-cp312-cp312-manylinux_2_28_x86_64.whl (1.8 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.8/1.8 MB 97.6 MB/s eta 0:00:00 Downloading cuda_bindings-13.2.0-cp312-cp312-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (6.3 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.3/6.3 MB 149.2 MB/s eta 0:00:00 Installing collected packages: torchaudio, nvidia-cusparselt-cu13, cuda-toolkit, nvidia-nvtx, nvidia-nvshmem-cu13, nvidia-nvjitlink, nvidia-nccl-cu13, nvidia-curand, nvidia-cufile, nvidia-cuda-runtime, nvidia-cuda-nvrtc, nvidia-cuda-cupti, nvidia-cublas, cuda-bindings, nvidia-cusparse, nvidia-cufft, nvidia-cudnn-cu13, nvidia-cusolver, torch, transformers, torchvision, bitsandbytes Attempting uninstall: torchaudio Found existing installation: torchaudio 2.10.0+cu128 Uninstalling torchaudio-2.10.0+cu128: Successfully uninstalled torchaudio-2.10.0+cu128 Attempting uninstall: cuda-toolkit Found existing installation: cuda-toolkit 12.8.1 Uninstalling cuda-toolkit-12.8.1: Successfully uninstalled cuda-toolkit-12.8.1 Attempting uninstall: cuda-bindings Found existing installation: cuda-bindings 12.9.4 Uninstalling cuda-bindings-12.9.4: Successfully uninstalled cuda-bindings-12.9.4 Attempting uninstall: torch Found existing installation: torch 2.10.0+cu128 Uninstalling torch-2.10.0+cu128: Successfully uninstalled torch-2.10.0+cu128 Attempting uninstall: transformers Found existing installation: transformers 5.0.0 Uninstalling transformers-5.0.0: Successfully uninstalled transformers-5.0.0 Attempting uninstall: torchvision Found existing installation: torchvision 0.25.0+cu128 Uninstalling torchvision-0.25.0+cu128: Successfully uninstalled torchvision-0.25.0+cu128 ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. cuda-python 12.9.4 requires cuda-bindings~=12.9.4, but you have cuda-bindings 13.2.0 which is incompatible. libcuml-cu12 26.2.0 requires cuda-toolkit[cublas,cufft,curand,cusolver,cusparse]==12.*, but you have cuda-toolkit 13.0.2 which is incompatible. libraft-cu12 26.2.0 requires cuda-toolkit[cublas,curand,cusolver,cusparse]==12.*, but you have cuda-toolkit 13.0.2 which is incompatible. cudf-cu12 26.2.1 requires cuda-toolkit[nvcc,nvrtc]==12.*, but you have cuda-toolkit 13.0.2 which is incompatible. libcuvs-cu12 26.2.0 requires cuda-toolkit[cublas,curand,cusolver,cusparse]==12.*, but you have cuda-toolkit 13.0.2 which is incompatible. cuml-cu12 26.2.0 requires cuda-toolkit[cublas,cufft,curand,cusolver,cusparse]==12.*, but you have cuda-toolkit 13.0.2 which is incompatible. Successfully installed bitsandbytes-0.49.2 cuda-bindings-13.2.0 cuda-toolkit-13.0.2 nvidia-cublas-13.1.0.3 nvidia-cuda-cupti-13.0.85 nvidia-cuda-nvrtc-13.0.88 nvidia-cuda-runtime-13.0.96 nvidia-cudnn-cu13-9.19.0.56 nvidia-cufft-12.0.0.61 nvidia-cufile-1.15.1.6 nvidia-curand-10.4.0.35 nvidia-cusolver-12.0.4.66 nvidia-cusparse-12.6.3.3 nvidia-cusparselt-cu13-0.8.0 nvidia-nccl-cu13-2.28.9 nvidia-nvjitlink-13.0.88 nvidia-nvshmem-cu13-3.4.5 nvidia-nvtx-13.0.85 torch-2.11.0 torchaudio-2.11.0 torchvision-0.26.0 transformers-5.3.0cell 2:
import torch, os, sys print("torch:", torch.__version__) print("cuda available:", torch.cuda.is_available()) print("torch cuda:", torch.version.cuda) !python -m bitsandbytes !nvidia-smicell 2 results:
torch: 2.11.0+cu130 cuda available: True torch cuda: 13.0 bitsandbytes library load error: libnvJitLink.so.13: cannot open shared object file: No such file or directory Traceback (most recent call last): File "/usr/local/lib/python3.12/dist-packages/bitsandbytes/cextension.py", line 320, in <module> lib = get_native_library() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/dist-packages/bitsandbytes/cextension.py", line 298, in get_native_library dll = ct.cdll.LoadLibrary(str(binary_path)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/ctypes/__init__.py", line 460, in LoadLibrary return self._dlltype(name) ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/ctypes/__init__.py", line 379, in __init__ self._handle = _dlopen(self._name, mode) ^^^^^^^^^^^^^^^^^^^^^^^^^ OSError: libnvJitLink.so.13: cannot open shared object file: No such file or directory =================== bitsandbytes v0.49.2 =================== Platform: Linux-6.6.113+-x86_64-with-glibc2.35 libc: glibc-2.35 Python: 3.12.12 PyTorch: 2.11.0+cu130 CUDA: 13.0 HIP: N/A XPU: N/A Related packages: accelerate: 1.13.0 diffusers: 0.37.0 numpy: 2.0.2 pip: 24.1.2 peft: 0.18.1 safetensors: 0.7.0 transformers: 5.3.0 triton: 3.6.0 trl: not found ============================================================ PyTorch settings found: CUDA_VERSION=130, Highest Compute Capability: (7, 5). Checking that the library is importable and CUDA is callable... Traceback (most recent call last): File "<frozen runpy>", line 198, in _run_module_as_main File "<frozen runpy>", line 88, in _run_code File "/usr/local/lib/python3.12/dist-packages/bitsandbytes/__main__.py", line 4, in <module> main() File "/usr/local/lib/python3.12/dist-packages/bitsandbytes/diagnostics/main.py", line 107, in main raise e File "/usr/local/lib/python3.12/dist-packages/bitsandbytes/diagnostics/main.py", line 96, in main sanity_check() File "/usr/local/lib/python3.12/dist-packages/bitsandbytes/diagnostics/main.py", line 40, in sanity_check adam.step() File "/usr/local/lib/python3.12/dist-packages/torch/optim/optimizer.py", line 533, in wrapper out = func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/dist-packages/torch/utils/_contextlib.py", line 124, in decorate_context return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/dist-packages/bitsandbytes/optim/optimizer.py", line 328, in step self.update_step(group, p, gindex, pindex) File "/usr/local/lib/python3.12/dist-packages/torch/utils/_contextlib.py", line 124, in decorate_context return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/dist-packages/bitsandbytes/optim/optimizer.py", line 557, in update_step F.optimizer_update_32bit( File "/usr/local/lib/python3.12/dist-packages/bitsandbytes/functional.py", line 1235, in optimizer_update_32bit torch.ops.bitsandbytes.optimizer_update_32bit( File "/usr/local/lib/python3.12/dist-packages/torch/_ops.py", line 1269, in __call__ return self._op(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/dist-packages/torch/_compile.py", line 54, in inner return disable_fn(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/dist-packages/torch/_dynamo/eval_frame.py", line 1263, in _fn return fn(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/dist-packages/torch/library.py", line 751, in func_no_dynamo return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/dist-packages/bitsandbytes/backends/cuda/ops.py", line 654, in _optimizer_update_32bit_impl optim_func( File "/usr/local/lib/python3.12/dist-packages/bitsandbytes/cextension.py", line 269, in throw_on_call raise RuntimeError(f"{self.formatted_error}Native code method attempted to call: lib.{name}()") RuntimeError: 🚨 CUDA SETUP ERROR: Missing dependency: libnvJitLink.so.13 🚨 CUDA 13.x runtime libraries were not found in the LD_LIBRARY_PATH. To fix this, make sure that: 1. You have installed CUDA 13.x toolkit on your system 2. The CUDA runtime libraries are in your LD_LIBRARY_PATH You can add them with (and persist the change by adding the line to your .bashrc): export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/path/to/cuda-13.x/ lib64 Original error: libnvJitLink.so.13: cannot open shared object file: No such file or directory 🔍 Run this command for detailed diagnostics: python -m bitsandbytes If you've tried everything and still have issues: 1. Include ALL version info (operating system, bitsandbytes, pytorch, cuda, python) 2. Describe what you've tried in detail 3. Open an issue with this information: https://github.com/bitsandbytes-foundation/bitsandbytes/issues Native code method attempted to call: lib.cadam32bit_grad_fp32() Tue Mar 24 05:50:05 2026 +-----------------------------------------------------------------------------------------+ | NVIDIA-SMI 580.82.07 Driver Version: 580.82.07 CUDA Version: 13.0 | +-----------------------------------------+------------------------+----------------------+ | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |=========================================+========================+======================| | 0 Tesla T4 Off | 00000000:00:04.0 Off | 0 | | N/A 46C P0 28W / 70W | 3MiB / 15360MiB | 3% Default | | | | N/A | +-----------------------------------------+------------------------+----------------------+ +-----------------------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=========================================================================================| | No running processes found | +-----------------------------------------------------------------------------------------+More context:
Bitsandbytes was causing problems before this. I was having version mismatch issues at first. I saw comments saying to use the most updated versions for bitsandbytes but that was not fixing my issues. I had to go through multiple rounds of installations to get to this cell 1 version which worked for a while (3 months). It was very ad-hoc I must say.
But now my environment is acting up again. I'm probably in a different container now and so I am running into this new issue with bitsandbytes.
From what I can gather, this is a lower level issue with CUDA and interfacing with bitsandbytes. I don't have much control over the environment so the only thing I can do is version changes, right?
I would love all of your help. Thanks in advance.
