DiffusionGemma on Ubuntu 26.04@WSL2

DiffusionGemmaUbuntu26.04WSL2.md

DiffusionGemma on Ubuntu 26.04@WSL2

This article is based on the following article:

• DiffusionGemma - How to Run Locally

Preparing uv for Python environment

curl -LsSf https://astral.sh/uv/install.sh | sh

See Installing uv for details.

Install CUDA Toolkit 13.3

wget https://developer.download.nvidia.com/compute/cuda/repos/wsl-ubuntu/x86_64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo apt-get update
sudo apt-get -y install cuda-toolkit-13-3

llama.cpp build

Based on the tutorial here:

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
gh pr checkout 24423
# NOTE: Need to specify nvcc path explicitly
cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_COMPILER=/usr/local/cuda/bin/nvcc -DCUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda
cmake --build build -j --config Release --target llama-diffusion-cli

Download the model from Hugging Face

uvx hf auth login

uvx hf download unsloth/diffusiongemma-26B-A4B-it-GGUF \
    --local-dir unsloth/diffusiongemma-26B-A4B-it-GGUF \
    --include "*Q4_K_M*"

See here for more details.

Now Let's get started

See Chat with DiffusionGemma