
Can LLMs Write Fast Multi-GPU Kernels? — Simran Arora, Together AI
Simran Arora explains how communication between GPUs constrains AI workloads and presents ParallelKittens primitives for writing efficient multi-GPU kernels. She then introduces ParallelKernelBench, an 87-problem evaluation, reporting that tested models produce some useful kernels but struggle with communication ordering, data partitioning, scheduling, and…
Simran Arora