rahulunair

This API documentation is designed for participants of the Advent of Multi-Modal Hackathon to help you access and utilize state-of-the-art AI models seamlessly. The APIs provide capabilities for generative text, multimodal inputs (image + text), and advanced image generation. Using the instructions and examples provided below, participants can integrate these models into their projects and build innovative, fully automated solutions for the hackathon challenges.

To deploy these models with a dedicated API for your project, explore our AI templates at https://tiberaicommunity.github.io/ai-templates. We can also assist in providing dedicated hardware upon request. For additional model suggestions or contributions, visit xpu_tgi or xpu_ray. If these resources were helpful, please consider leaving a star on the project repositories to show your support!

**LLava-Next

rahulunair

#!/bin/bash

model="NousResearch/Nous-Hermes-Llama2-13b"
volume="$PWD/data"
tgi_version="2.0.0" # https://github.com/huggingface/tgi-gaudi/releases/tag/v2.0.0
max_input_token=16000
max_total_token=32000
container_name="tgi-container"

kill_existing_container() {

rahulunair

Quick Guide: Running FLUX Schnell Model on Habana

Step 1: Pull and run the Habana PyTorch Docker image with the necessary configurations

docker run -it --runtime=habana -e HABANA_VISIBLE_DEVICES=all --ipc=host --cap-add=sys_nice --ulimit memlock=-1:-1 --security-opt seccomp=unconfined \
-v /home/ubuntu/workspace:/workspace \
-v ~/.cache/huggingface:/root/.cache/huggingface \
vault.habana.ai/gaudi-docker/1.18.0/ubuntu24.04/habanalabs/pytorch-installer-2.4.0:latest /bin/bash

rahulunair

rahulunair

rahulunair

import time

import torch
import torch.nn as nn
import torch.optim as optim
from torchvision import datasets, transforms, models
from torch.utils.data import DataLoader

if torch.__version__ < '2.4':
    try:

rahulunair

import logging
import warnings
import argparse
import time
import numpy as np

import torch
import torch.nn as nn
import torchvision.transforms as transforms
import torchvision.datasets as datasets

rahulunair

import logging
import warnings

logging.basicConfig(level=logging.ERROR)
st_logger = logging.getLogger("sentence_transformers")
st_logger.setLevel(logging.ERROR)
warnings.filterwarnings("ignore")

import argparse
import time

rahulunair

import warnings
warnings.simplefilter(action="ignore")

from transformers.utils import logging
logging.set_verbosity_error()

import torch
import intel_extension_for_pytorch as ipex
from transformers import AutoModelForCausalLM, AutoTokenizer

rahulunair

#!/bin/bash

apt-get purge -y intel-basekit intel-aikit intel-hpckit intel-renderkit

# add keys
wget -O- https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB \
| gpg --dearmor | tee /usr/share/keyrings/oneapi-archive-keyring.gpg > /dev/null

echo "deb [signed-by=/usr/share/keyrings/oneapi-archive-keyring.gpg] https://apt.repos.intel.com/oneapi all main" | tee /etc/apt/sources.list.d/oneAPI.list