Inference

In this post, I’ll walk you through deploying DeepSeek-V4-Flash-DSpark — a 284B-parameter MoE model (13B active) — across two NVIDIA DGX Sparks running as EKS Hybrid Nodes, connected over a ConnectX-7 RoCE stacking link. A single DGX Spark has a 128 GB unified memory pool, so the weights alone (~167 GB) don’t fit on one node. Spreading the model across two Sparks with tensor parallelism (TP=2) leaves just enough room for the weights, the KV cache, and framework overhead. I’ll show you how to reach 50–60 tok/s decode with a 1.79M KV pool — a 512K context window at >3× concurrency as deployed, or up to a single 1M context stream. ...