DGX-Spark

In this post, I’ll walk you through deploying DeepSeek-V4-Flash-DSpark — a 284B-parameter MoE model (13B active) — across two NVIDIA DGX Sparks running as EKS Hybrid Nodes, connected over a ConnectX-7 RoCE stacking link. A single DGX Spark has a 128 GB unified memory pool, so the weights alone (~167 GB) don’t fit on one node. Spreading the model across two Sparks with tensor parallelism (TP=2) leaves just enough room for the weights, the KV cache, and framework overhead. I’ll show you how to reach 50–60 tok/s decode with a 1.79M KV pool — a 512K context window at >3× concurrency as deployed, or up to a single 1M context stream. ...

I’ve been running a single DGX Spark (GB10) as an EKS Hybrid Node for a while, and recently a second one landed on my desk. The plan is to stack the two Sparks with one QSFP112 cable to create a little two-node EKS Hybrid cluster for distributed inference. This post documents how to set up RoCE/RDMA networking and perform NCCL testing across the two Spark nodes, including all lessons learned. ...