Hi, I’m Sheng

In this post, I’ll walk you through deploying DeepSeek-V4-Flash-DSpark — a 284B-parameter MoE model (13B active) — across two NVIDIA DGX Sparks running as EKS Hybrid Nodes, connected over a ConnectX-7 RoCE stacking link. A single DGX Spark has a 128 GB unified memory pool, so the weights alone (~167 GB) don’t fit on one node. Spreading the model across two Sparks with tensor parallelism (TP=2) leaves just enough room for the weights, the KV cache, and framework overhead. I’ll show you how to reach 50–60 tok/s decode with a 1.79M KV pool — a 512K context window at >3× concurrency as deployed, or up to a single 1M context stream. ...

I’ve been running a single DGX Spark (GB10) as an EKS Hybrid Node for a while, and recently a second one landed on my desk. The plan is to stack the two Sparks with one QSFP112 cable to create a little two-node EKS Hybrid cluster for distributed inference. This post documents how to set up RoCE/RDMA networking and perform NCCL testing across the two Spark nodes, including all lessons learned. ...

I’ve recently been working on an AI-at-the-edge project (SafetyLens) — that’s a topic for another post. During the demo development, I frequently needed to tweak the model serving engine, swap models in and out, and refine configurations. Every one of those changes meant restarting vLLM, and each restart triggered a full cold start that took five minutes or more before the server was ready to serve again. And that’s with just a single Qwen3.6-35B-A3B model running off local NVMe storage on 1x NVIDIA DGX Spark node. This will only get worse once I scale the deployment to a multi-node cluster backed by an external PVC, where the weights have to travel over the network on every start. So I went looking for ways to cut down the vLLM cold-start time — and it turns out most of it is recoverable, with no changes to the model or its output quality. ...

This is the 5th episode of our NKE lab series. In this episode, I will demonstrate how you can easily build a fully-automated GitOps continues delivery (CD) pipeline, by using Github, NKE and Argo CD. GitOps is a operational framework that takes DevOps best practices (such as version control, Infra-as-Code, CI/CD etc), and applies them to modern and cloud native infrastructure such as Kubernetes-based clusters. There are two GitOps approaches: Push-based and Pull-based, and you can reach more about each model at here. This post will focus on the Pull-based approach as it provides many benefits such as better version control and governance, more automation and self-service capabilities, and easier for rollback, auditing/compliance suitable for large and stable production environment. ...

This is the 4th episode of our NKE lab series. Previously, I have demonstrated how you can easily deploy a NKE cluster in a Nutanix CE lab environment, and I have explored some NKE platform features including out-of-the-box CSI and CNI support. In this episode, we’ll take a look how you can accelerate Kubernetes application development by integrating NKE with Nutanix Database Service (NDB). NDB is a Database-as-a-Service designed to help developers speed up application development and simplify database administration across on-prem and public clouds. It simplifies database operations such as test DB provisioning/cloning and integrated snapshots/backup etc. It also provides a consistent “Database-as-Code” experience using REST API and K8s integrations. NDB supports most popular database engines, and you can read more about it at here. ...

This is the 3rd episode of our NKE lab series. Previously, I have walked through: How to deploy a NKE-enabled Kubernetes cluster in a nested Nutanix CE environment How to provide persistent storage to your NKE clusters using 2x Nutanix CSI options In this episode, we’ll deep dive into the NKE networking spaces by exploring the following: PART-1: Exploring Calico CNI deployment models within a NKE cluster PART-2: Applying standard Kubernetes network policy in a NKE cluster PART-3: Leveraging Calico specific policies in a NKE cluster pre-requisites a 1-node or 3-node Nutanix CE 2.0 cluster deployed in nested virtualization depending on your lab compute capacity, as documented here and here a NKE-enabled K8s cluster deployed in Nutanix CE (see Ep1) a Guestbook demo app deployed onto the NKE cluster (see Ep2) a lab network environment supports VLAN tagging and provides basic infra services such as AD, DNS, NTP etc (these are required when installing the CE cluster) a Linux/Mac workstation for managing the Kubernetes cluster, with Kubectl installed PART-1: Exploring Calico CNI models in NKE Calico is recognized as the most popular CNI plugins within he Kubernetes community, and it has been widely deployed in production thanks to its reliable performance and comprehensive networking and security features. ...

This is the 2nd episode of our NKE lab series. In the 1st episode, I have demonstrated how you can easily deploy an enterprise-grade NKE cluster in a Nutanix CE lab environment with nested virtualization. In this episode, we’ll deploy a containerized multi-tier web application onto our NKE cluster, by leveraging the built-in Nutanix CSI driver to provide persistent storage for the demo app. Specifically, we’ll explore 2x Nutanix CSI options: ...

This blog is the 1st episode of a Nutanix Kubernetes Engine (NKE) home lab series. In this post, I will walk through the detailed process of deploying an enterprise-ready NKE-enabled Kubernetes cluster within a Nutanix CE environment. Nutanix CE is a free version of Nutanix AOS, which powers the Nutanix Enterprise Cloud Platform. It is designed for people interested in test driving Nutanix platform features and capabilities in a non-production or PoC environment. Even better, Nutanix CE also works in a nested virtualization deployment on top of ESXi/vSphere. This makes it perfect for hands-on testing or exploring in a safe environment such as home-lab, which is exactly what I’m running here! ...

With the recent release of VMware Cloud on AWS SDDC version 1.18, we have introduced a ton of advanced networking capabilities which opened up possibilities for many new interesting use cases. Customers can now utilise the NSX Manager UI (or VMC Policy API) to configure route aggregation at each SDDC level, and this provides an efficient way to solve the 100 DX route limit. Customer can also create additional Tier-1 Compute Gateways (Multi-CGWs) with static route injection capabilities to address different requirements such as network multi-tenancy, overlapping IPv4 environments and integrating with 3rd-party network & security appliances etc. You can read more details about the new features at here. ...

With the recently announced Amazon FSx for NetApp ONTAP, it is very exciting that for the first time we have a fully managed ONTAP file system in the cloud! What’s more interesting about this service is that we can now deliver high-performance block storage to the workloads running on VMware Cloud on AWS (VMC) through a first-party Amazon managed service! In this post I will walk you through a simple example for provisioning and integrating iSCSI-based block storage to a Windows workload running on VMC environment using Amazon FSx for NetAPP ONTAP. For this demo I’ve provisioned the FSx service in a shared service VPC, which is connected to the VMC SDDC cluster through an AWS Transit Gateway (TGW) via VPN attachment (as per below diagram). ...