According to this link, you need to install the latest CUDA to make the GPU accessible.
So follow the official instruction to install CUDA 12.8, and install pytorch-nightly later:
pip install --pre torch --index-url https://download.pytorch.org/whl/nightly/cu128
I had to install the open version of the NVIDIA driver 570 (not the proprietary since it is behind the open version).
So I did:
sudo apt install nvidia-driver-570-open
and the GPU (an ASUS Prime 5070 - without Ti) is recognized by Ubuntu 24.04