GPT4All fails to load CUDA backend on RTX 2050, kompute device not working

I'm trying to use GPU acceleration with the GPT4All Python library but I can't get it to work despite having a compatible NVIDIA GPU.

Environment:

GPU: NVIDIA GeForce RTX 2050 (4GB VRAM)

CUDA: 13.1 (verified with nvcc --version)

Driver: 591.86

OS: Windows 11

GPT4All version: 3.10.0

Python: 3.13.5

Model: Meta-Llama-3-8B-Instruct.Q4_0.gguf

Problem:

When I try to use device='gpu' or device='cuda':

python

gpt = GPT4All(model_path, device='gpu')

I get these errors:

Failed to load llamamodel-mainline-cuda-avxonly.dll: LoadLibraryExW failed with error 0x7e Failed to load llamamodel-mainline-cuda.dll: LoadLibraryExW failed with error 0x7e constructGlobalLlama: could not find Llama implementation for backend: cuda

What I tried:

GPT4All.list_gpus() returns ['kompute:NVIDIA GeForce RTX 2050'] — so the GPU is detected.

Then I tried:

python

gpt = GPT4All(model_path, device='kompute') # and gpt = GPT4All(model_path, device='kompute:NVIDIA GeForce RTX 2050')

Both still show the same CUDA DLL errors and fall back to CPU.

I also tried adding the CUDA bin directory manually:

python

import os os.add_dll_directory(r"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.1\bin")

Still the same result.

Question:

How can I get GPT4All to actually use my GPU via kompute? Are the CUDA DLL errors blocking kompute from loading, or are they just warnings? Is there a missing dependency I need to install?