Justin Chu justinchuby

NXRT Scheduling Research: Dynamic Session/Model Scheduling in ML Inference Runtimes

Date: 2026-07-15
Purpose: Survey existing implementations relevant to NXRT's scheduling/session management design.

1. vLLM — Session Preemption & KV Cache Swap

What It Does

Flightdeck 2 Governance And Communication Review

Executive Summary

Flightdeck already has the right execution substrate for multi-agent work: clear role separation, role-scoped tool permissions, a typed task DAG, reviewer handoff, and persistent status/reporting surfaces. The main gap versus Paperclip is not orchestration power. It is operational governance.

Paperclip's strongest transferable patterns for Flightdeck are:

First-class approval and interaction objects rather than implied "gated" states.
Explicit blocker ownership and unblock actions rather than blocked tasks with only dependency/state visibility.

Paperclip 观察笔记

这是一份中文速记，整理自一次对 Paperclip 的实际使用和代码/文档阅读。它不是官方文档复述，而是站在使用者和工程实现视角，对这个系统“怎么工作”以及“哪里还不够顺”做的总结。

一句话概括

Paperclip 本质上不是一个单纯的聊天壳，而是一个以 issue + heartbeat 为核心的 agent 控制平面。Agent 不靠持续在线的会话工作，而是被事件唤醒，在一个有明确边界的任务上下文里执行，再把结果写回到 issue、评论、文档、blocker 或子任务里。

它大概是怎么工作的

Qwen3.5 Gated DeltaNet Linear Attention — Technical Analysis

Purpose: Research for proposing new ONNX operators to support Qwen3.5/Qwen3-Next linear attention.

Date: 2026-02-27

Sources:

HuggingFace transformers: src/transformers/models/qwen3_next/modular_qwen3_next.py
HuggingFace transformers: src/transformers/models/qwen3_5/modular_qwen3_5.py
flash-linear-attention library: fla/ops/gated_delta_rule/

Symbolic Shape Inference for ONNX IR

Problem Statement

Design and implement symbolic shape inference capability for the ONNX IR that can propagate shapes through the graph while preserving symbolic dimension relationships (e.g., batch, seq_len, N+1).

Current State Analysis

Existing Infrastructure

SymbolicDim class (_core.py:1241): Immutable symbolic dimension with string or None values
Shape class (_core.py:1330): Supports mixed static/dynamic dimensions, freezing, denotations

	#!/usr/bin/env python
	# Copyright (c) ONNX Project Contributors
	# SPDX-License-Identifier: Apache-2.0

	r"""Phi-4 Multimodal output parity check: ONNX (mobius) vs HuggingFace.

	Compares the prefill logits of the ONNX 4-model pipeline (vision,
	speech, embedding, decoder) against HuggingFace's PyTorch reference
	for every modality combination:

	# Mask-specific variables
	# TODO: Reconcile differences between `seqlens_k` and `key_total_seq_lens` in the GroupQueryAttention and SparseAttention implementations. Ideally the same subgraph can be shared for both.
	self.mask_attrs = {
	# "mask_name": "", # Name of node that outputs 4D causal attention mask (used as add_qk in MultiHeadAttention)
	"seqlens_k": "", # Sum of each row in attention mask - 1 (used as input to GroupQueryAttention)
	"total_seq_len": "", # Size of total sequence length in attention mask (used as input to GroupQueryAttention and SparseAttention)
	# "block_row_indices": "", # Row indices of CSR format of block mask (used as input to SparseAttention)
	# "block_col_indices": "", # Col indices of CSR format of block mask (used as input to SparseAttention)
	# "key_total_seq_lens": "", # Sum of each row in attention mask (used as input to SparseAttention)
	}

	"""
	Perch v2 Bird Audio Classification Model - PyTorch Implementation

	This module provides a pure PyTorch implementation of the Perch v2 model,
	converted from the original ONNX model. The model processes 10-second audio
	clips at 16kHz (160,000 samples) and produces bird species embeddings and
	classification logits.

	Architecture:
	- Learned frontend with convolution-based STFT and mel filterbank

	import onnx_ir as ir
	import onnx_ir.passes.common
	import onnxscript

	m = ir.load("perch_v2_opt3.onnx")

	for node in m.graph:
	if node.op_type == "MatMul":
	print(node)
	if node.inputs[0].producer().op_type == "Reshape":