AI ENGINEER SHANGHAI 2026
Lena Onyshchenko
PyTorch Compiler Engineer
NVIDIA
Talk
Async or Nothing: When TMA Pays Off in torch.compile
Rapid Fire · 10 min · Ten minutes, one concrete problem, engineering lesson or demo. Handovers are included in each slot; conversations continue in the breaks.
November 5 · 17:15-17:25 · Main stage →
Shanghai time (UTC+8) · Tentative program; changes will be communicated.
Every NVIDIA GPU generation since Hopper ships TMA, a dedicated engine that copies whole tiles into shared memory while the cores keep computing. Turn it on for every `torch.compile` kernel and SiLU on GB200 gets 1.8x slower. This talk builds the mental model that explains why: when a kernel is memory-bound, what an async copy needs in order to pay off, and the hardware rules that quietly change your tiling. Then it shows how Inductor can stop guessing and instead race TMA and non-TMA versions of each kernel on first launch, the way it already does for GEMMs, without making compile time worse. You leave knowing which of your kernels TMA can speed up, which it can't, and how to tell from a profile in a minute.