Skip to content

Instantly share code, notes, and snippets.

@hradec
Last active June 10, 2025 18:17
Show Gist options
  • Save hradec/32840097fe4cada3119468d8027155f9 to your computer and use it in GitHub Desktop.
Save hradec/32840097fe4cada3119468d8027155f9 to your computer and use it in GitHub Desktop.
A simple way to setup a python virtual environment without having to install anything (NO CONDA) - Linux and Windows
#!/bin/bash
# ================================================================================================================================================================================================================================
# a simple script to create a virtual environment for python, install some basic packages and run a python script
# I known anaconda does the same, but I prefer to have a self-contained python environment in a folder without
# the need to install anything in the system.
# This script automatically downloads a self-contained python distribution in windows, so it can be used in any
# computer without any installation.
#
# In windows, use python.cmd to run this script. python.cmd will automatically download MSYS2 with bash and
# all dependencies to properly run this script.
# https://gist.github.com/hradec/370dc46326256ed64ff58c7f59446765#file-python-cmd
#
# For AMD, install rocm (apt install -y rocm* ; apt install -y hipcc) and use --check-rocm to check if it's working
#
# TODO: env vars for versions of python and cuda.
# ================================================================================================================================================================================================================================
# Windows always use standalone python distribution
# Force to use standalone python distribution in linux
# comment the LINUX_STANDALONE_URL= line to use system python in linux
#➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖
# user configurable variables to setup versions of python, cuda and pytorch
#➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖
# python 3.10
# ================================================================================================================
LINUX_STANDALONE_URL='https://github.com/indygreg/python-build-standalone/releases/download/20241016/cpython-3.10.15+20241016-x86_64-unknown-linux-gnu-pgo+lto-full.tar.zst'
WINDS_STANDALONE_URL='https://github.com/winpython/winpython/releases/download/6.1.20230527/Winpython64-3.10.11.1dot.exe'
# these versions work with python 3.10 in windows
# torch 2.1.0 / cuda 12.1.0
# version 2.5.1/12.4 is broken on python 3.10 in windows
# ================================================================================================================
# python 3.11
# ================================================================================================================
LINUX_STANDALONE_URL='https://github.com/indygreg/python-build-standalone/releases/download/20241016/cpython-3.11.10+20241016-x86_64-unknown-linux-gnu-pgo+lto-full.tar.zst'
WINDS_STANDALONE_URL='https://github.com/winpython/winpython/releases/download/7.1.20240203final/Winpython64-3.11.8.0dot.exe'
# torch and cuda versions for windows
WINDS_TORCH_VERSION=2.1.0
WINDS_CUDA_VERSION=12.1.0
# torch and cuda versions for linux
LINUX_TORCH_VERSION=2.5.1
LINUX_CUDA_VERSION=12.4.0
# ================================================================================================================
# python 3.12 (https://winpython.github.io/)
# ================================================================================================================
# LINUX_STANDALONE_URL='https://github.com/astral-sh/python-build-standalone/releases/download/20250521/cpython-3.12.10+20250521-x86_64-unknown-linux-gnu-pgo+lto-full.tar.zst'
# WINDS_STANDALONE_URL='https://github.com/winpython/winpython/releases/download/15.3.20250425final/Winpython64-3.12.10.0dot.exe'
# torch and cuda versions for windows
# WINDS_TORCH_VERSION=2.7.0
# WINDS_CUDA_VERSION=12.6.0
# torch and cuda versions for linux
# LINUX_TORCH_VERSION=2.7.0
# LINUX_CUDA_VERSION=12.6.0
#➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖
# end of user configurable variables
#➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖
# ================================================================================================================
# procedurally generated variables based on user configurable variables and the current system
# ================================================================================================================
# check if we are in windows( $CYGWIN = 1 ) or linux( $CYGWIN = 0 )
export CYGWIN="0"
export OSNAME="linux"
[ "$(grep CYGWIN /proc/version)" != "" ] && export CYGWIN="1"
[ "$(grep MSYS_NT /proc/version)" != "" ] && export CYGWIN="1"
[ "$CYGWIN" == "1" ] && export OSNAME="win32"
# set different versions for windows and linux
if [ "$CYGWIN" == "1" ] ; then
TORCH_VERSION=$WINDS_TORCH_VERSION
CUDA_VERSION=$WINDS_CUDA_VERSION
else
TORCH_VERSION=$LINUX_TORCH_VERSION
CUDA_VERSION=$LINUX_CUDA_VERSION
fi
# get the first two numbers of the cuda version for the pytorch url installation
CUDA_VERSION_URL=$(echo $CUDA_VERSION | awk -F'.' '{print $1$2}')
# get the script folder
SRC=$(dirname "$(readlink -f $0)")
if [ "$CYGWIN" == "1" ] ; then
SRC="$(cygpath -m -a $BASH_SOURCE | sed 's#/[^/]*$##')"
fi
SRC=$(echo "$SRC" | sed 's/ /\ /g')
# get the current folder
export PWD=$(pwd -P | sed 's/ /\ /g')
export __PWD=$(pwd -P | sed 's/ /\ /g')
# check if cuda folder exists where this script is
# if so, set CUDA_HOME to that folder
if [ -e "$SRC"/cuda/$CUDA_VERSION ] ; then
export CUDA_HOME="$SRC"/cuda/$CUDA_VERSION/$OSNAME/
export CUDA_TOOLKIT_ROOT_DIR="$CUDA_HOME"
fi
# detect the version of cuda pytorch was installed with, just in case!
if [ "$CYGWIN" == "1" ] ; then
export __CUDA_VERSION=$(ls -dt "$SRC"/cuda/$(grep 'cuda:' "$SRC"/WP*/python*/Lib/site-packages/torch/version.py 2>/dev/null | awk -F'=' '{print $(NF)}' | sed -e "s/'//g" -e "s/ //g" | tr -d '\r')* 2>/dev/null | head -1)
else
export __CUDA_VERSION=$(ls -dt "$SRC"/cuda/$(grep 'cuda:' "$SRC"/LP*/install/lib/python*/site-packages/torch/version.py 2>/dev/null | awk -F'=' '{print $(NF)}' | sed -e "s/'//g" -e "s/ //g" | tr -d '\r')* 2>/dev/null | head -1)
fi
# if the cuda version exists, set the environment variables
if [ "$__CUDA_VERSION" != "" ] ; then
export CUDA_VERSION=$(basename $__CUDA_VERSION)
export CUDA_HOME="$SRC/cuda/$CUDA_VERSION/$OSNAME/"
export CUDA_TOOLKIT_ROOT_DIR="$CUDA_HOME"
fi
# change the home folder to the script folder so everything stays
# self-contained in the script folder
export USERPROFILE="$SRC/home/"
export HOME="$SRC/home/"
mkdir -p $HOME
# check if nvidia or amd rocm is installed in the system
export CUDA_NVIDIA_INSTALL=0
if [ "$(which nvidia-smi 2>/dev/null)" != "" ] ; then
export CUDA_NVIDIA_INSTALL=1
fi
export CUDA_AMD_INSTALL=0
if [ "$(which rocm-smi 2>/dev/null)" != "" ] ; then
export CUDA_AMD_INSTALL=1
fi
# display help!!
if [ "$1" == "-h" ] ; then
echo "Usage: python.sh [--check-cuda] [--check-rocm] [--bash] [--cmd] [ --force-cuda-cpu-install ] [ --force-cuda-nvidia-install ] [ --force-cuda-amd-install ] | <python standard arguments>"
exit 0
fi
export FORCE_CUDA_INSTALL=0
if [ "$1" == "--force-cuda-cpu-install" ] ; then
shift
export FORCE_CUDA_INSTALL=1
export CUDA_NVIDIA_INSTALL=0
export CUDA_AMD_INSTALL=0
fi
if [ "$1" == "--force-cuda-nvidia-install" ] ; then
shift
export FORCE_CUDA_INSTALL=1
export CUDA_NVIDIA_INSTALL=1
fi
if [ "$1" == "--force-cuda-amd-install" ] ; then
shift
export FORCE_CUDA_INSTALL=1
export CUDA_AMD_INSTALL=1
fi
# reset python environment so we don't inherit anything from the system
unset PYTHONPATH
unset PYTHONHOME
unset PYTHONDONTWRITEBYTECODE
# pip is a flag to trigger default pip package installation
export pip=0
# the path separator for linux or windows
export sep=":"
# system python executable by default
virtualenv_python="/usr/bin/python3"
# ================================================================================================================
# in windows, always download a self contained full python distribution and store in the script folder
# ================================================================================================================
if [ "$CYGWIN" == "1" ] ; then
export sep=";"
# download winpython in windows, so python is self-contained in the folder
if [ "$(ls -d "$SRC"/WP*)" == "" ] ; then
cd "$SRC"
curl -L -k "$WINDS_STANDALONE_URL" -o winpython.7z
7z x winpython.7z && rm -f winpython.7z
cd "$PWD"
export pip=1
fi
wpython_folder=$(ls -drt "$SRC"/WP* | tail -1)
wpython_folder2=$(ls -drt "$wpython_folder"/python* | tail -1)
virtualenv_python="$wpython_folder2/python"
# we need ";:' befora adding $PATH to PATH, or else it wont convert the first
# path in PATH to windows format. The conversion relies on regex ':\/.\/' (ex :/c/ => ;C:)
# to identify if a path should be converted to windows format.
# we also convert python folder and cuda_home to windows format, to prevent then to be
# converted to UNC (//IP ADDRESS//SHARE NAME) format in case they are already in windows format
export PATH=";;$(cygpath -a -w $(readlink -f $wpython_folder2));;$(cygpath -a -w $(readlink -f $wpython_folder2/Scripts));;$(cygpath -a -w $CUDA_HOME/bin/);:$PATH"
export PATH=";;$(cygpath -w $(readlink -f $wpython_folder2));;$(cygpath -w $(readlink -f $wpython_folder2/Scripts));;$(cygpath -w $CUDA_HOME/bin/);:$PATH"
# ================================================================================================================
# in linux, download a self-contained python distribution if LINUX_STANDALONE_URL is set
# ================================================================================================================
elif [ "$LINUX_STANDALONE_URL" != "" ] ; then
VERSION=$(echo "$LINUX_STANDALONE_URL" | awk -F 'cpython-' '{print $2}' | awk -F'+' '{print $1}')
# download a self contained python distribution in linux, if none is already downloaded
if [ "$(ls -d "$SRC"/LPy64-$VERSION)" == "" ] ; then
# curl -L -k 'https://github.com/25077667/standalone-python/releases/download/release-2024-04-29/release-3.12-x86_64.tar.gz' | tar -xz -f -
cd "$SRC"
curl -L -k "$LINUX_STANDALONE_URL" | tar --zstd -x -f -
# use the same name convention as in windows
mv "$SRC"/python "$SRC"/LPy64-$VERSION
cd "$PWD"
export pip=1
fi
wpython_folder=$(ls -drt "$SRC"/LP* | tail -1)
wpython_folder2="$wpython_folder/install"
virtualenv_python="$wpython_folder2/bin/python"
export PATH="$wpython_folder2$sep$CUDA_HOME/bin/$sep$PATH"
# ================================================================================================================
# not windows/not LINUX_STANDALONE_URL, use virtualenv to create a custom self contained environment (not tested)
# ================================================================================================================
else
# get python version and hostname
pv=$("$virtualenv_python" -c 'import sys,socket;print(f"{sys.version.split()[0]}-{socket.gethostname()}")' | head -1)
# create virtual environment folder name
venv="$SRC"/env.$pv
if [ "$CYGWIN" == "1" ] ; then
pv=$("$virtualenv_python" -c 'import sys,socket;print(f"{sys.version.split()[0]}-win")' | head -1)
venv="$SRC"/env.$pv
venv="$(cygpath -a -m "$venv")"
fi
# initialize the virtual environment, if none is already created
if [ ! -e "$SRC"/env.$pv/bin/activate ] || [ "$REINSTALL" != "" ] ; then
cd "$SRC"
if [ "$CYGWIN" == "0" ] ; then
"$virtualenv_python" -m ensurepip
fi
"$virtualenv_python" -m pip install virtualenv
"$virtualenv_python" -m virtualenv -p python "$venv"
[ $? -ne 0 ] && exit 1
if [ "$CYGWIN" == "1" ] ; then
ln -s Scripts "$venv"/bin
ln -s python "$venv"/bin/python3.exe
else
[ ! -e "$venv"/bin/python ] && ln -s "$venv"/bin/python3 "$venv"/bin/python
fi
source "$SRC"/env.$pv/bin/activate
cd "$PWD"
export pip=1
else
source "$SRC"/env.$pv/bin/activate
fi
export LD_LIBRARY_PATH="$SRC"/libs/$sep$LD_LIBRARY_PATH
export PYTHONPATH=$PYTHONPATH$sep"$SRC"/src/
export PYTHONPATH="$SRC"$sep$PYTHONPATH
export PYTHONPATH="$__PWD"/python$sep$PYTHONPATH
export virtualenv_python="$SRC/env.$pv/bin/python"
fi
# ================================================================================================================
# install some default packages - it looks for requirements.txt file in the script folder, in the parent
# folder (../) or in the ../python folder. If none is found, it installs some basic packages, including pytorch
# ================================================================================================================
if [ "$pip" == "1" ] ; then
# this ensures pip works correctly in virtualenv for some reason (also fixes certificate problems)
curl -k -sS https://bootstrap.pypa.io/get-pip.py | "$virtualenv_python"
# update certificates so pip works correctly
"$virtualenv_python" -m pip install --upgrade certifi
# upgrade pip
"$virtualenv_python" -m pip install --upgrade pip
if [ -e "$SRC"/requirements.txt ] ; then
"$virtualenv_python" -m pip install -r "$SRC"/requirements.txt
elif [ -e "$SRC"/../requirements.txt ] ; then
"$virtualenv_python" -m pip install -r "$SRC"/../requirements.txt
elif [ -e "$SRC"/../python/requirements.txt ] ; then
"$virtualenv_python" -m pip install -r "$SRC"/../python/requirements.txt
else
# install some basic packages
"$virtualenv_python" -m pip install PySide6
"$virtualenv_python" -m pip install numpy
"$virtualenv_python" -m pip install pandas
"$virtualenv_python" -m pip install opencv-python
"$virtualenv_python" -m pip install pyinstaller
"$virtualenv_python" -m pip install scipy
"$virtualenv_python" -m pip install matplotlib
export FORCE_CUDA_INSTALL=1
fi
fi
# ================================================================================================================
# pytorch installation
# ================================================================================================================
if [ "$FORCE_CUDA_INSTALL" == "1" ] ; then
"$virtualenv_python" -m pip uninstall -y torch torchvision torchaudio
# if amd rocm in the system
if [ "$CUDA_AMD_INSTALL" == "1" ] ; then
# install pytorch with rocm support (rocm 6.2 = rocm62), according with the website table: https://pytorch.org/
# $SRC/env.$pv/bin/python -m pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/rocm6.2
"$virtualenv_python" -m pip install torch==1.13.1+rocm5.2 torchvision==0.14.1+rocm5.2 torchaudio==0.13.1 --extra-index-url https://download.pytorch.org/whl/rocm5.2
# install support for AMD APUs to allocate system memory as GPU memory
cd "$SRC"/env.$pv/
git clone https://github.com/pomoke/torch-apu-helper.git
cd torch-apu-helper
hipcc ./gttalloc.c -o alloc.so -shared -fPIC
install_path=$($SRC/env.$pv/bin/python -c 'import torch,os;print(os.path.dirname(torch.__file__))')
cp ./alloc.so "$install_path"/
# if nvidia in the system
elif [ "$CUDA_NVIDIA_INSTALL" == "1" ] ; then
# install pytorch with cuda support (cuda 12.1 = cu121), according with the website table: https://pytorch.org/
"$virtualenv_python" -m pip install torch==$TORCH_VERSION torchvision torchaudio --index-url https://download.pytorch.org/whl/cu$CUDA_VERSION_URL
else
"$virtualenv_python" -m pip install torch torchvision torchaudio
fi
# install pytorch with intel gpu support (https://pytorch.org/docs/stable/notes/get_start_xpu.html)
# "$virtualenv_python" -m pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/xpu
fi
# ================================================================================================================
# all environment is set, so from here we can run stuff
# ================================================================================================================
cd $__PWD
# run custom environment variables in the same folder as python.sh is
if [ -e $SRC/.env ] ; then
source $SRC/.env
fi
# run custom environment variables in the current folder, if any exists.
if [ -e $__PWD/.env ] ; then
source $__PWD/.env
fi
if [ "$1" == "--bash" ] ; then
shift
echo $PATH
if [ "$CYGWIN" == "1" ] ; then
"$SRC/msys64/usr/bin/bash" --noprofile --norc --verbose "$@"
else
bash --noprofile --norc --verbose "$@"
fi
exit 0
elif [ "$1" == "--cmd" ] ; then
shift
echo $PATH
cmd "$@"
exit 0
elif [ "$1" == "--check-cuda" ] ; then
"$virtualenv_python" -c 'import torch;print(f"torch path:{torch.__file__}");print(f"torch version: {torch.version.__version__}");print(f"torch cuda version: {torch.version.cuda}");print(torch.cuda.get_device_properties(0).total_memory)'
elif [ "$1" == "--check-rocm" ] ; then
"$virtualenv_python" -c 'import torch,os;new_alloc = torch.cuda.memory.CUDAPluggableAllocator(os.path.dirname(torch.__file__)+"/alloc.so","gtt_alloc","gtt_free");torch.cuda.memory.change_current_allocator(new_alloc);print(f"torch path:{torch.__file__}");print(f"torch version: {torch.version.__version__}");print(f"torch rocm version: {torch.version.hip}");print(torch.cuda.get_device_properties(0).total_memory)'
else
"$virtualenv_python" "$@"
fi
@hradec
Copy link
Author

hradec commented Feb 23, 2025

The script will check the installed pytorch cuda version, and set the CUDA_VERSION and CUDA_HOME env vars accordingly.
TODO: download and extract cuda sdk automatically.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment