AI ENGINEER SHANGHAI 2026
Chen Wang
Head of Partner Engineering
Baseten
Talk
Full-Stack Inference Engineering: Latency, Throughput and Cost
Rapid Fire · 10 min · Ten minutes, one concrete problem, engineering lesson or demo. Handovers are included in each slot; conversations continue in the breaks.
November 5 · 11:10-11:20 · Main stage →
Shanghai time (UTC+8) · Tentative program; changes will be communicated.
Open-source model inference is becoming a compelling option for enterprises, especially those building AI-native applications. Delivering fast, efficient, and reliable inference requires engineering across the full stack, from CUDA kernels and Kubernetes orchestration to quantization, caching, and disaggregation. This talk explores the key design decisions and emerging techniques shaping production inference systems, with practical insights into balancing latency, throughput, cost, and reliability.