Fine Tuning LLM - GTX 1060 - WSL2-Ubuntu-2022

gistfile1.txt

1. Ubuntu 2020 -> default python 3.8  ->  require pyenv to handle higher versions of python
2. Ubuntu 2024 -> default python 3.12 -> PyTorch does not currently provide prebuilt wheels for Python 3.12 (cp312) via the official website or PyPI.

3. Installed Ubunt 2022 -> default python 3.10 


wsl --install -d Ubuntu-22.04

wsl --setdefault Ubuntu-22.04

sudo apt install python3-pip
sudo apt install python3-venv

sudo pip install pipx

pipx install uv

https://developer.nvidia.com/cuda-12-6-0-download-archive?target_os=Linux&target_arch=x86_64&Distribution=WSL-Ubuntu&target_version=2.0&target_type=deb_local

wget https://developer.download.nvidia.com/compute/cuda/repos/wsl-ubuntu/x86_64/cuda-wsl-ubuntu.pin
sudo mv cuda-wsl-ubuntu.pin /etc/apt/preferences.d/cuda-repository-pin-600
wget https://developer.download.nvidia.com/compute/cuda/12.6.0/local_installers/cuda-repo-wsl-ubuntu-12-6-local_12.6.0-1_amd64.deb
sudo dpkg -i cuda-repo-wsl-ubuntu-12-6-local_12.6.0-1_amd64.deb
sudo cp /var/cuda-repo-wsl-ubuntu-12-6-local/cuda-*-keyring.gpg /usr/share/keyrings/
sudo apt-get update
sudo apt-get -y install cuda-toolkit-12-6

export PATH=/usr/local/cuda-12.6/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda-12.6/lib64:$LD_LIBRARY_PATH
export PATH="/usr/bin:$PATH"
export PATH="$HOME/.local/bin:$PATH"

nvcc --version
nvidia-smi

if used uv venv, make sure to install uv pip instead of just pip.. 

python3 -m venv .venv
source .venv/bin/activate

pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
python -c "import torch; print(torch.cuda.is_available()); print(torch.cuda.get_device_name(0))"
pip install scikit-learn matplotlib pandas jupyterlab notebook
pip install transformers peft bitsandbytes accelerate

No officially released version of Unsloth currently supports the GTX 1060, as the software requires a minimum CUDA Compute Capability of 7.0 for GPU acceleration, while the GTX 1060 only provides Compute Capability 6.1. Unsloth's documentation and PyPI page make it explicit that support starts from GPUs such as the GTX 1070 and later (Volta, Turing, Ampere, Ada, and Blackwell architectures). Attempts to use Unsloth with GTX 1060 result in errors from triton or PyTorch kernels stating, "Triton only supports devices of CUDA Capability >= 7.0, but your device is of CUDA capability 6.1"

---

GTX 1060 and Unsloth
Unsloth requires CUDA Compute Capability 7.0 or higher for GPU acceleration.

GTX 1060 = Compute Capability 6.1.

Software, kernels, and Triton in Unsloth will refuse to run with 6.1:
You’ll get errors like:

text
Triton only supports devices of CUDA Capability >= 7.0, but your device is of CUDA capability 6.1
Supported GPUs are GTX 1070 and above (Volta, Turing, Ampere, Ada, Blackwell).