Good question! I am collecting human data on how quantization affects outputs. See here for more information: ggml-org/llama.cpp#5962
In the meantime, use the largest that fully fits in your GPU. If you can comfortably fit Q4_K_S, try using a model with more parameters.
See the wiki upstream: https://github.com/ggerganov/llama.cpp/wiki/Feature-matrix
This guide is based on multiple guides as well as official instructions for the other boards found on the Internet:
I have gone through all these steps recently and got a working board with my favorite Arch Linux ARM. I hope it will be helpful for someone else.
| #btrfs benchmark for daily used desktop OS |
Since we're dealing with simpler AV1 encoding, that does mean we'll be eskewing aomenc-av1, since it requires 2-pass encoding to be able to take advantage of it, and I use it externally since I have access to a special build.
Now, let's get on with the simple guide.
You'll first need to be reasonably competent with command line builds or use a recent up to date ffmpeg GUI with support for SVT-AV1.
As such, I would recommend getting a master git ffmpeg build for the operating system of your choice right here:
Hello. I've decided to share a lot more of my knowledge in public forums from now on, and to not divert any of my focus away from improving the world in a way that stays written in history.
This Gist is about discussing on how to improve AV1 decoding performance on 2 fronts: improving performance through more efficient decoding, and through decoding aware encoding.
Here are many tips on how to improve decoding performance on any machine:
I have two Github accounts: oanhnn (personal) and superman (for work). I want to use both accounts on same computer (without typing password everytime, when doing git push or pull).
Use ssh keys and define host aliases in ssh config file (each alias for an account).
This is a cheat sheet for how to perform various actions to ZSH, which can be tricky to find on the web as the syntax is not intuitive and it is generally not very well-documented.
| Description | Syntax |
|---|---|
| Get the length of a string | ${#VARNAME} |
| Get a single character | ${VARNAME[index]} |
| Computer Information: | |
| Manufacturer: Unknown | |
| Model: Unknown | |
| Form Factor: Desktop | |
| No Touch Input Detected | |
| Processor Information: | |
| CPU Vendor: AuthenticAMD | |
| CPU Brand: AMD Ryzen 7 1700 Eight-Core Processor | |
| CPU Family: 0x17 |
