Compression at the Edge
Chris Alexiuk · Daniel Han · Asma Beevi · Merve Noyan · Parth Sareen
AI Engineer World's Fair 2026 · 46:01
← All organizationsunsloth.ai ↗
AI model training and local inference
Unsloth
Unsloth builds open-source software for developers to train, fine-tune and run AI models locally. Its product family includes Unsloth Core, the code-based version; Unsloth Studio, a web interface; and Unsloth Desktop, available for Windows, macOS and Linux. Users can customize language, embedding and audio models, generate images and videos, and build training datasets from documents. The training tools support reinforcement learning, LoRA, QLoRA and full fine-tuning, with export formats for deployment.
Brothers Daniel Han and Michael Han founded Unsloth in 2023, and Daniel serves as CEO. Its engineering work includes custom kernels for model training, including Triton kernels for rotary position embeddings and multilayer perceptrons, alongside batching algorithms for longer-context reinforcement learning. The kernels, combined with padding-free training and packing, speed up training and reduce GPU memory use.
Its software supports CPUs and GPUs from multiple vendors, including NVIDIA, AMD, Intel and Apple Silicon. An OpenAI-compatible API lets developers serve local models to other applications, while coding-agent connections let users work with local models through tools such as Claude Code and Codex. Its GitHub repository had about 75,000 stars as of August 2026.
5 talks
Newest first
Special topics in Kernels, RL, Reward Hacking in Agents
AI Engineer World's Fair 2026 · 2:20:21
Advanced: Reinforcement Learning, Kernels, Reasoning, Quantization & Agents — Daniel Han
AI Engineer World's Fair 2025 · 2:42:28
1 speaker at AIE
Affiliations reflect their AIE appearances, not necessarily current employment.
Start here
- Advanced: Reinforcement Learning, Kernels, Reasoning, Quantization & Agents — Daniel Han
Start with the essay based on Daniel Han’s talk to understand training stages, PPO’s additional models, and the memory constraints of a practical GRPO experiment.
Daniel HanAI Engineer World's Fair 2025
- Fixing bugs in Gemma, Llama & Phi-3
Use this talk to learn how duplicated BOS tokens and padding/EOS choices can lead to reduced accuracy, NaN gradients, or endless generation.
Daniel HanAI Engineer World's Fair 2024
- Special topics in Kernels, RL, Reward Hacking in Agents
Choose this workshop for its examination of SWE-bench reliability and its comparison of software optimization, specialized hardware, and GPU-kernel development.
Daniel HanAI Engineer World's Fair 2026
- Low Level Technicals of LLMs
Unsloth's Daniel Han leads an interactive workshop on low-level language-model internals and practical debugging.
Daniel HanAI Engineer World's Fair 2024
Messages from the stage
Small implementation errors, substantial failures
Han traces fine-tuning failures to token handling, chat templates, and embeddings. His low-level workshop supplies supporting foundations in tokenization, embedding lookup, and attention matrices.
Rewards need to measure the intended task
Han’s reinforcement learning workshop explains why passing partial checks can fall short of producing a good result. The essay adaptation connects that problem to verifiable rewards and practical GRPO training.
Compression requires quality checks
Han’s later workshop examines dynamic low-bit quantization and cost-performance tradeoffs. The shared edge panel discusses layer-sensitive compression, long-context evaluation, and benchmarking compressed models.
Affiliations reflect each recorded session, not necessarily current employment. Compression at the Edge is a joint panel with participants from NVIDIA, Unsloth, Hugging Face, and Ollama.