Distillation (machine learning)

Distillation in machine learning is a technique where a large, complex model that has learned to solve a problem is used to teach a smaller, simpler model to do the same thing. Think of it like having an expert distill their knowledge into a concise guidebook that a student can carry around and learn from more easily. This makes AI systems faster and more practical to use in real-world applications without sacrificing much accuracy.