Building a Slurm GPU Cluster on AWS — and Fine-Tuning Qwen3-4B Across It
Slurm is the workload manager that runs most of the world’s HPC and large-scale ML training. This post walks through building a small Slurm GPU cluster on AWS from scratch, and runs a distributed LoRA fine-tune of Qwen3-4B across two GPU nodes (8 GPUs total). Why Slurm for training If you already run Kubernetes, it’s worth being clear about why you’d reach for Slurm at all. Kubernetes is a service orchestrator — built to keep long-running things alive and healthy. A training run is the opposite shape: a finite job that grabs N GPUs, runs to completion, and exits. That’s a batch workload, and Slurm is a batch scheduler built exactly for it — with a real queue, gang scheduling (all your nodes start together or the job waits), and one-line multi-node launch via srun. ...