EGAD entropy-guided adaptive distillation focuses knowledge transfer on high-uncertainty tokens

EGAD introduces entropy-guided adaptive distillation that weights token-level knowledge transfer by uncertainty rather than treating all tokens equally in LLM knowledge distillation. The approach improves efficiency and downstream performance in resource-constrained deployment scenarios.