Reference to get around

iTerm downloads

Run this on remote to get it2* utilities: curl -sS https://webi.sh/iterm2-utils | sh && source ~/.iterm2_shell_integration.bash

Screen

screen -S <session_name>
screen -ls
screen -r <attach_to_session>
Ctrl + A and then D to detach
screen -X -S <screen session> quit
# Enabled scrolling in screen. Add following lines in ~/.screenrc
# Enable mouse scrolling and scroll bar history scrolling
termcapinfo xterm* ti@:te@

Docker

Install docker https://www.digitalocean.com/community/tutorials/how-to-install-and-use-docker-on-ubuntu-20-04

# Running containers
docker ps
# Copying files
docker cp containerId:source_path destination_path
# Access other cmd as user
sudo chmod 666 /var/run/docker.sock

Tmux

Rsync

rsync -avz hostname:/file/path .

SSH

To keep SSH connection alive

# Add this to ~/.ssh/config
Host *
    ServerAliveInterval 240
    ServerAliveCountMax 2

Remote access setup

chmod 700 ~/.ssh
# add id_rsa.pub from server to ~/.ssh/authorized_keys on remote
chmod 600 ~/.ssh/authorized_keys

To debug key issues, check keys that are already added with ssh-add -l

VSCode

Kill vscode without sudo on remote ps aux | grep .vscode-server | awk '{print $2}' | xargs kill

AWK

Extract between multiple lines matching a start and an end pattern: bash awk '/start/,/end/' <file>

Initial config on new VM

sudo apt update && sudo apt upgrade -y && sudo apt install -y git vim-gtk3 build-essential binutils python3-dev

Python

sudo apt instal python3-pip python3.12-venv
python3 -m venv ~/base
source ~/base/bin/activate
python3 -m pip install --upgrade pip
pip install numpy pandas matplotlib

Switch Python versions:

ls /usr/bin/python*
sudo update-alternatives --list python
sudo update-alternatives --install /usr/bin/python python /usr/bin/python2 1
sudo update-alternatives --install /usr/bin/python python /usr/bin/python3 2
sudo update-alternatives --config python

Conda

Get rid of InvalidSpec when working with Nvidia repos conda install -c conda-forge --override-channels

Git

# In order to switch to a remote branch, make sure to fetch your remote branch with “git fetch” first. You can then switch to it by executing “git checkout” with the “-t” option and the name of the branch.
$ git fetch
$ git checkout -t <remote_name>/<branch_name>

# Moving repos around
git remote set-url origin <new-repo-url>
git push

# Undo git add
git reset

# Undo git commit before push
git reset --soft --HEAD~2

# Move git repo retaining history
git clone https://sourceRepoURL
cd sourceRepoFolder
git push --mirror https://targetRepoURL

# Create a standalone patch
git add . 
git commit -m <message>
git format-patch -1

Linux stuff

wget: Unable to resolve host address xxx wget --no-proxy --no-check-certificate
Always use ls --color=auto to avoid issues with weird characters when piping ls into a file
Copy ssh to remote server ssh-copy-id -i ~/.ssh/id_rsa.pub login@serverIP

Enable bash on login servers

if [ -f ~/.bashrc ]; then
  . ~/.bashrc
fi

Redirect all streams to file ./test > log 2>&1

Slurm

sinfo: list of partitions

Nvidia profiling

sudo -E env PATH=$PATH ncu --metrics lts__t_request_hit_rate --log-file output ./bfs.out ../../data/bfs/graph65536.txt

ML Environment

Windows

To add a new env to jupyter

# Create a new env
$ conda create -n new_env -y
# Install ipykernel and add the env
$ pip install ipykernel
$ ipython kernel install --user --name=new_env
# Remove an environment
jupyter kernelspec uninstall env_name

To enter jupyer "jupyter lab --no-browser" on WSL2

rajesh-s/reference.md