NCCL

Slurm is the workload manager that runs most of the world’s HPC and large-scale ML training. This post walks through building a small Slurm GPU cluster on AWS from scratch, and runs a distributed LoRA fine-tune of Qwen3-4B across two GPU nodes (8 GPUs total). Why Slurm for training If you already run Kubernetes, it’s worth being clear about why you’d reach for Slurm at all. Kubernetes is a service orchestrator — built to keep long-running things alive and healthy. A training run is the opposite shape: a finite job that grabs N GPUs, runs to completion, and exits. That’s a batch workload, and Slurm is a batch scheduler built exactly for it — with a real queue, gang scheduling (all your nodes start together or the job waits), and one-line multi-node launch via srun. ...

I’ve been running a single DGX Spark (GB10) as an EKS Hybrid Node for a while, and recently a second one landed on my desk. The plan is to stack the two Sparks with one QSFP112 cable to create a little two-node EKS Hybrid cluster for distributed inference. This post documents how to set up RoCE/RDMA networking and perform NCCL testing across the two Spark nodes, including all lessons learned. ...