Knowledge Distillation
Temperature T:
4
α (distill weight):
0.70
Distill
Reset
Student learns from soft targets (teacher logits) — dark knowledge transfers inter-class similarities