What is distillation?

Training one AI model to copy useful behavior from another, often larger, model. This can make AI cheaper or faster to run, but some abilities may be lost.

Also called Model distillation, Knowledge distillation.

For example: A smaller model learns from answers produced by a larger one.