So that is why I changed above cuda:11.8.0 to cuda:11.6.0 and to. First nvidia-smi shows CUDA Version: 11.6. I am trying to create the docker container in windows with the above dockerfile. So for the example above Cuda needs >= 11.8. On Ubuntu at least, I had to make sure that my local Cuda version was the same or above the one installed with the Docker Image. So for everyone who is also working on running the package on Docker. RUN pip install -no-cache-dir torch torchvision torchaudio torchviz -extra-index-url ĬMD nvidia-smi & python3 -m bitsandbytes I'm not familiar with pulling together packages of this nature however, if you'd be willing to provide me with some guidance and tolerate inevitable blunders I'd make that would likely cause some chuckles in the PRs, I'd happily apply my mind to a few fixes here?įROM nvidia/cuda:11.8.0-cudnn8-devel-ubuntu20.04 I may be wrong however, that is certainly my expectation. Unfortunately though, I think that consumers of the package expect that dependency handling as well as the telemetry provided could and should be improved. I say that with the full understanding that the issue appears to be caused by differences in how the Nvidia components are installed and made available to the OS via paths and symlinks. To my mind, the path handling issue is a bug. If I may say so, I don't believe that closing the issue makes sense, in my opinion, given that the behvaiour from the software in this instance creates instability. My solution is primarily for Windows (WSL), yes however, it can be used wherever the path handling issue described causes difficulty. Hi my apologies for the delayed response. Hopefully the solution helps everyone else. Thank you to everyone who contributed ideas on this thread, they helped me figure out what was wrong and formulate a solution that works for me. Once my distro had bounced, my code worked. I then used wsl -shutdown from cmd on Windows to shutdown Ubuntu (it starts right back up when you run wsl). In my case, I added /usr/local/cuda/lib64 to the path. Once that is done, use vim to edit /etc/environment and add the directory where you just created the symlink, to the path. Within that directory, use ln -s to create a link to libcuda.so in the same directory as libcudart.so. In my case, I have libcuda.so in the WSL directory and libcudart.so in a symlinked directory called /usr/local/cuda/lib64. You'll use this to locate libcuda.so and libcudart.so using the locate command. ![]() To fix the problem, install plocate using sudo apt-get install plocate. The warning is technically true but quite misleading without context. ![]() As in, the bitsandbytes package cannot find the CUDA dependencies in the path environment variable, falls back to the CPU library in its package, which then causes the not compiled for GPU issue. The issue is caused by a dependency management problem related to paths, which itself is exacerbated by the CPU fallback code and the misleading message that results. I've read through the thread and would like to provide both an explanation and a solution that should work for everyone affected. Not sure why is so hard to use a tool made for CUDA on a CUDA-enabled machine, is there a specific reason? I'd love to help but I am a noob at this stuff Either pass torch_dtype=torch.float16 or don't pass this argument at all to remove this warning. Overriding torch_dtype=None with `torch_dtype=torch.float16` due to requirements of `bitsandbytes` to enable model loading in mixed int8. Warn("The installed version of bitsandbytes was compiled without GPU support. 8-bit optimizers and GPU quantization are unavailable. opt/conda/lib/python3.10/site-packages/bitsandbytes/cextension.py:31: UserWarning: The installed version of bitsandbytes was compiled without GPU support. ![]() opt/conda/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: No libcudart.so found! Install CUDA or the cudatoolkit package (anaconda)!ĬUDA SETUP: Highest compute capability among GPUs detected: 8.6ĬUDA SETUP: Loading binary /opt/conda/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cpu.so. opt/conda/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: | GPU GI CI PID Type Process name GPU Memory | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. ![]() | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |