Model compression: quantization-aware training, distillation and more

Explain the main ways to make a model smaller and faster for deployment: quantization (post-training versus quantization-aware training), knowledge distillation and pruning. For each, say how it works, what it costs in accuracy and when to use it. Be ready for related basics, such as what model.eval() changes and how contrastive learning works.