Skip to content

Instantly share code, notes, and snippets.

import torch
from optimum.quanto import QuantizedDiffusersModel, freeze, qfloat8, quantize
from diffusers import FluxPipeline, FluxTransformer2DModel
class QuantizedFluxTransformer2DModel(QuantizedDiffusersModel):
base_class = FluxTransformer2DModel
@sayakpaul
sayakpaul / inference_with_torchao_serialized.py
Last active January 13, 2025 01:51
Shows how to run Flux schnell under 17GBs without bells and whistles. It additionally shows how to serialize the quantized checkpoint and load it back.
import torch
from huggingface_hub import hf_hub_download
from diffusers import FluxTransformer2DModel, DiffusionPipeline
dtype, device = torch.bfloat16, "cuda"
ckpt_id = "black-forest-labs/FLUX.1-schnell"
with torch.device("meta"):
config = FluxTransformer2DModel.load_config(ckpt_id, subfolder="transformer")
model = FluxTransformer2DModel.from_config(config).to(dtype)
@sayakpaul
sayakpaul / distributed_inference_diffusers.py
Last active September 10, 2024 02:04
Minimal example to show how to run distributed inference from a set of prompts with diffusers and accelerate.
# Originally by jiwooya1000, put together together by sayakpaul.
# Documentation: https://huggingface.co/docs/diffusers/main/en/training/distributed_inference
"""
Run:
accelerate launch distributed_inference_diffusers.py --batch_size 8
# Enable memory optimizations for large models like SD3
accelerate launch distributed_inference_diffusers.py --batch_size 8 --low_mem=1
@crasm
crasm / gguf-merge.sh
Last active August 17, 2024 01:12
Shell script for merging TheBloke's .gguf-split model files
#!/bin/sh
log() {
format="$1"; shift
# shellcheck disable=SC2059
>&2 printf "$format\n" "$@"
}
usage() {
>&2 cat <<EOF
@al-swaiti
al-swaiti / improve_fonts.md
Created September 5, 2023 13:35 — forked from YoEight/improve_fonts.md
Improve fonts archlinux

Improve Fonts

Newest

Make your Arch fonts beautiful easily! This is what I do when I install Arch Linux to improve the fonts.

You may consider the following settings to improve your fonts for system-wide usage without installing a patched font library packages (eg. Infinality):

Install some fonts, for example:
sudo pacman -S ttf-dejavu ttf-liberation noto-fonts

@RabeyaMuna
RabeyaMuna / Q4.py
Created May 7, 2020 02:43
The parent_directory function returns the name of the directory that's located just above the current working directory. Remember that '..' is a relative path alias that means "go up to the parent directory". Fill in the gaps to complete this function.
import os
def parent_directory():
# Create a relative path to the parent
# of the current working directory
relative_parent = os.path.join(os.getcwd(), os.pardir)
# Return the absolute path of the parent directory
return os.path.abspath(relative_parent)
print(parent_directory())