New method traces data membership in self-distillation models
Researchers developed OP-Print to identify if specific training examples were used in on-policy self-distillation.
Researchers created OP-Print to perform membership inference attacks. This method works against on-policy self-distillation.
On-policy distillation uses the student's own outputs as supervision. This teacher-free design is called self-distillation.
The supervision imprint is sparse and structured. It concentrates on uncertainty-sensitive correction sites.
In gray-box settings, OP-Print aggregates the privileged teacher-student residual correction. In black-box settings, it measures restart-based output-basin stability.
The study investigates gray-box and black-box threat models. Using two coupled views of the same training example may strengthen memorization.
Key facts
- OP-Print is a unified attack framework for on-policy self-distillation.
- The supervision imprint concentrates on uncertainty-sensitive correction sites.
- Baseline attacks remain at random guessing even under gray-box access.
- OP-Print is effective in both gray-box and black-box settings.