Xiaodong Zeng

How can useful AI run directly on phones, PCs and other personal devices rather than depending on the cloud? This talk explores memory bottlenecks in on-device models, using early expert routing and flash streaming to reduce the resident memory footprint. It also covers combining millisecond-scale local decisions with larger on-device models, with examples from headphones, robots and smart glasses.