plan.txt

None selected 

Skip to content
Using Gmail with screen readers

1 of 143,628
DGX Spark pilot proposal for Rubin NVL72
Inbox

Alec Ashe-Cunningham <aleccunningham96@gmail.com>
9:16 AM (0 minutes ago)
to me

I propose a 60-day pilot using three DGX Spark systems to build reproducible workflows for model serving, benchmarking, and release testing in preparation for our upcoming Rubin NVL72 deployment.

The goal is to resolve application integration and serving-configuration issues in a dedicated environment, then bring the tested workflows and evaluation tools to the shared infrastructure. I would lead the pilot, publish weekly results, and document the system so that another engineer could operate it.

System	Primary use	Team benefit
Spark 1	A stable, tested vLLM endpoint for an agreed internal pilot	Application developers can integrate with the endpoint without being interrupted by runtime experiments.
Spark 2	Testing candidate model versions, runtime upgrades, quantization, and performance changes	The team can compare changes with a stable baseline before promoting them.
Spark 3	A second worker for distributed inference with Spark 2, or an independent replica	The team can test model sharding, routing, failure recovery, and scaling while Spark 1 remains available.
Load generation and report collection would run on existing infrastructure.

The third system would allow us to run two-node experiments without taking down the reference service. With only two systems, we would need to schedule interruptions or suspend distributed testing whenever the reference endpoint was needed.

The pilot would also provide a structured path for me to develop inference-serving expertise that the team can reuse. I would study request scheduling, prefill and decode, KV-cache behavior, quantization, and distributed-serving strategies, then test each topic against the agreed workload. Weekly results would explain not just which configuration performed better, but why, under what conditions, and whether output quality changed. I would turn those findings into short operating notes and review them with a second engineer. The Spark systems would support the learning and experiments; they would not be treated as a performance proxy for NVL72.

I propose the following deliverables, measured from the date the hardware becomes available:

Milestone	Deliverable	Acceptance evidence
Day 10	Reproducible deployment and reference endpoint	Pinned software and model versions, launch automation, successful requests from one pilot application, and a short explanation of the serving path and key memory constraints.
Day 20	Benchmarking and task-quality evaluation tools	Measurements of latency, throughput, errors, and memory usage across agreed prompt lengths and concurrency levels; approximately 50 task-specific regression cases; and an analysis of observed bottlenecks.
Day 30	Distributed-serving and recovery report	A comparison of one server, two independent replicas, and a supported two-node sharded configuration, with documented placement trade-offs, worker-failure behavior, and rollback results.
Day 45	Release-evaluation workflow	Candidate changes tested against the reference configuration using clear quality and performance criteria for promotion, with written explanations of accepted and rejected changes.
Day 60	Operational handoff and recommendation on whether to continue	A second engineer successfully reproduces a deployment and benchmark, and a report identifies useful configurations, remaining limitations, actual demand, and questions to validate on NVL72.
At the start of the pilot, we would select one real internal workload and identify its owner. Possible examples include a coding-agent backend or a document-processing workflow. We would also agree on acceptable output quality and latency. These are potential examples, not assumed requirements.

The benchmark reports would measure time to first token, time between generated tokens, request-completion latency, error rates, and throughput while meeting the agreed latency targets. An improvement would count only if task quality remained acceptable. We would also record negative results so that the team would not repeat unsuccessful experiments.

For NVL72, the reusable outputs would include workload definitions, evaluation cases, deployment procedures, monitoring, operating instructions, and a documented approach to investigating serving bottlenecks. NVL72 would still require its own supported software environment, hardware acceptance tests, placement decisions, and performance measurements. Results from the Spark systems would not be presented as estimates of NVL72’s capacity.

The pilot would require approximately 10 hours of my time per week, including hands-on experiments, structured study, analysis, and documentation; access to existing load-generation infrastructure; and support from a second engineer for review and handoff. The proposed procurement budget is $18,000 before tax for three systems and compatible interconnect cables, subject to actual vendor quotes and delivery dates. This is a planning limit, not a confirmed price.

If delivery is delayed beyond NVL72’s arrival, I would begin developing the evaluation tools and operating procedures on available compute. The Spark systems would still need to justify their cost as an ongoing environment for regression testing and experimentation.

At the end of the 60-day pilot, we would review the completed deliverables, independent team usage, time spent maintaining the lab, and how often we actually needed both the reference service and two-node testing at the same time. If the third system had little demonstrated value, we would reassign it and avoid further expansion.