Lyken17

Env setup

For python project, you should first follow the README to setup. If not specified, then use uv to create local env for dev. If it requires specific cuda version, or depends on library like (verl,sglang,vllm), then use conda instead.

SLURM Cluster Setup

Detect whether you are running on a SLURM cluster

which srun

Lyken17

PYTHONUNBUFFERED=1 \
    python3 -m verl.trainer.main_ppo \
    algorithm.adv_estimator=grpo \
    data.train_files="<your data>" \
    data.val_files="<your val>" \
    data.filter_overlong_prompts=True \
    data.train_batch_size=128 \
    data.val_batch_size=512 \
    data.max_prompt_length=9216 \
    data.max_response_length=8192 \

Lyken17

root@batch-block1-3665:/home/ligengz/workspace/slime# bash scripts/run-qwen3-4B.sh

Did not find any active Ray processes.
+ export PYTHONBUFFERED=16
+ PYTHONBUFFERED=16
++ nvidia-smi topo -m
++ grep -o 'NV[0-9][0-9]*'
++ wc -l
+ NVLINK_COUNT=56
+ '[' 56 -gt 0 ']'

Lyken17

python -m sglang.launch_server \
    --model Qwen/qwen3-8B --port 8000 \
    --dtype auto --data-parallel-size 8 --tensor-parallel-size 1

##################################################################

lm_eval --model local-completions \
    --tasks gsm8k,gsm8k_cot,aime24,aime25 \
    --model_args model=Qwen/qwen3-8B,base_url=http://127.0.0.1:8000/v1/completions,num_concurrent=32,max_retries=3
    

Lyken17

// MAIN CONVERSATION
{"role": "system", "content": "You are Qwen, created, by alibaba cloud, you are a helpful assistant. You are a helpful assistant."}
{"role": "user", "content": "Find the prime from 1 to 10"}
{"role": "assistant", "content": "Okay, so I need to find the prime numbers between 1 and 10. Let's think in parallel <Parallel>\n<Goal>\n<Outline>\n1. Find the prime numbers between 1 and 5.</Outline>\n<Outline>\n2. Find the prime numbers between 6 and 10.</Outline>\n</Goal>\n"} //OUTPUT// Note: the end of the conversation is not <|im_end|> but rather </Goal>
{"role": "assistant", "content": "I will think about the prime numbers between 1 and 5 first. ....."}
{"role": "assistant", "content": "I will then think about the prime numbers between 6 and 10 ....."}
{"role": "assistant", "content": "After analyzing both ranges, the prime numbers between 1 and 10 are: 2, 3, 5, 7."} //OUTPUT

// THREAD CONVERSATION 1
{"role": "system", "content": "You are Qwen, created, by alibaba cloud, you are a helpful assi

Lyken17

import os
import json
import time
import concurrent.futures
from tqdm import tqdm
from google import genai
from google.genai import types
import fire

def load_video_info_from_jsonl(jsonl_path):

Lyken17

import pandas as pd
import time
from sglang import gen, system
import os, sys, os.path as osp
import asyncio
import openai
from tqdm import tqdm
import json
import sys
from sglang.utils import wait_for_server, print_highlight, terminate_process

Lyken17

ligengz@:~/workspace/VILA-dev$ python serving/lmsys_test.py     --model openai/nvila-8b-dev     --api-base http://localhost:8000     --req-per-sec 1
/home/ligengz/anaconda3/envs/hf/lib/python3.10/site-packages/transformers/tokenization_utils_base.py:1601: FutureWarning: `clean_up_tokenization_spaces` was not set. It will be set to `True` by default. This behavior will be depracted in transformers v4.45, and will be then set to `False` by default. For more details check this issue: https://github.com/huggingface/transformers/issues/31884
  warnings.warn(
/home/ligengz/anaconda3/envs/hf/lib/python3.10/site-packages/pydantic/main.py:347: UserWarning: Pydantic serializer warnings:
  Expected `str` but got `int` - serialized value may not be as expected
  return self.__pydantic_serializer__.to_python(

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Lyken17

// https://www.shj.work/tools/secha/
// cheat code for the small color testing game


function processPlates() {
    const d = {};
    const plates = document.querySelectorAll("#box span");
    
    plates.forEach(p => {
        if (!(p.style.backgroundColor in d)) {

Lyken17

from llava.wids import ShardListDataset

train_url = "https://storage.googleapis.com/webdataset/fake-imagenet/imagenet-train.json"
'''
{
  "__kind__": "wids-shard-index-v1",
  "wids_version": 1,
  "shardlist": [
    {