Tag: soft-targets

All the articles with the tag "soft-targets".

• Notes on: Distilling the Knowledge in a Neural Network

  6 Sep, 2026

  Hinton, Vinyals & Dean's 2015 paper — the mechanics of soft-target distillation, the "omit a digit" MNIST experiment, and how it scales an ensemble of specialists back into one model.