| # Power Usage Calculator for AI Workloads | |
| ''' | |
| # Serving | |
| $ vllm serve meta-llama/Llama-3.3-70B-Instruct --tensor-parallel-size 4 --num-scheduler-steps 20 --quantization=fp8 --gpu-memory-utilization=0.97 | |
| INFO 01-13 04:59:05 api_server.py:712] vLLM API server version 0.6.6.post2.dev5+g5ce4627a | |
| # Benchmark - we do bs=64 to emulate https://arxiv.org/pdf/2310.03003 | |
| cmd = [ | |
| "python", os.path.expanduser("~/vllm/benchmarks/benchmark_serving.py"), |
This guide is for homelab admins who understand IPv4s well but find setting up IPv6 hard or annoying because things work differently. In some ways, managing an IPv6 network can be simpler than IPv4, one just needs to learn some new concepts and discard some old ones.
Let’s begin.
First of all, there are some concepts that one must unlearn from ipv4:
Concept 1
This is a living document. Everything in this document is made in good faith of being accurate, but like I just said; we don't yet know everything about what's going on.
Update: I've disabled comments as of 2025-01-26 to avoid everyone having notifications for something a year on if someone wants to suggest a correction. Folks are free to email to suggest corrections still, of course.
This worked on 14/May/23. The instructions will probably require updating in the future.
llama is a text prediction model similar to GPT-2, and the version of GPT-3 that has not been fine tuned yet. It is also possible to run fine tuned versions (like alpaca or vicuna with this. I think. Those versions are more focused on answering questions)
Note: I have been told that this does not support multiple GPUs. It can only use a single GPU.
It is possible to run LLama 13B with a 6GB graphics card now! (e.g. a RTX 2060). Thanks to the amazing work involved in llama.cpp. The latest change is CUDA/cuBLAS which allows you pick an arbitrary number of the transformer layers to be run on the GPU. This is perfect for low VRAM.
08737ef720f0510c7ec2aa84d7f70c691073c35d.
| #!/bin/bash | |
| # whisper-stream.sh | |
| # | |
| # Take a url supported by yt-dlp, dump 30-second segments to the current | |
| # directory named by unix timestamp, and transcribe each segment using Whisper. | |
| # | |
| # example: TZ=Australia/Canberra ./whisper-stream.sh "https://..." | |
| # | |
| # The time displayed is the time when ffmpeg first opens the segment for |
| #!/usr/bin/env bash | |
| # IMPORTANT: this script is now part of my dotfiles and maintained there | |
| # see https://github.com/Diaoul/dotfiles/blob/main/.config/hypr/scripts/workspaces.sh | |
| set -e | |
| declare -i last_called=0 | |
| declare -i throttle_by=4 | |
| @throttle() { | |
| local -i now=$(date +%s) |
Note: I have moved this list to a proper repository. I'll leave this gist up, but it won't be updated. To submit an idea, open a PR on the repo.
Note that I have not tried all of these personally, and cannot and do not vouch for all of the tools listed here. In most cases, the descriptions here are copied directly from their code repos. Some may have been abandoned. Investigate before installing/using.
The ones I use regularly include: bat, dust, fd, fend, hyperfine, miniserve, ripgrep, just, cargo-audit and cargo-wipe.
Please comment below if you have an update, e.g., with another networking-related dataset.
| " let g:python_host_prog = '/usr/bin/python2' | |
| " let g:python3_host_prog = '/usr/bin/python3' | |
| " ************************************* | |
| " PLUGIN SECTION for Vim-Plug | |
| " ************************************* | |
| call plug#begin('~/.config/nvim/plugged') | |
| " Make sure you use single quotes |
This is a compiled list of falsehoods programmers tend to believe about working with time.
Don't re-invent a date time library yourself. If you think you understand everything about time, you're probably doing it wrong.
