LoRA

A few weeks ago I built a Slurm GPU cluster from scratch on bare EC2 and fine-tuned Qwen3-4B across two g6.12xlarge nodes. This post runs the same Slurm fine-tune — but using the Slinky operator on Amazon EKS. Slinky is an open-source project for running Slurm inside Kubernetes: a Kubernetes operator, with custom controllers and CRDs, that manages the lifecycle of a Slurm cluster and its NodeSets as native Kubernetes resources. The point is to get the best of both worlds: combining Slurm’s deterministic, fine-grained batch scheduling with Kubernetes’ dynamic resource allocation and rapid scaling. ...

Slurm is the workload manager that runs most of the world’s HPC and large-scale ML training. This post walks through building a small Slurm GPU cluster on AWS from scratch, and runs a distributed LoRA fine-tune of Qwen3-4B across two GPU nodes (8 GPUs total). Why Slurm for training If you already run Kubernetes, it’s worth being clear about why you’d reach for Slurm at all. Kubernetes is a service orchestrator — built to keep long-running things alive and healthy. A training run is the opposite shape: a finite job that grabs N GPUs, runs to completion, and exits. That’s a batch workload, and Slurm is a batch scheduler built exactly for it — with a real queue, gang scheduling (all your nodes start together or the job waits), and one-line multi-node launch via srun. ...