New method protects face privacy from image generation models
A new framework called SEAL modifies only 5% of facial image locations to disrupt identity acquisition.
Text-to-image diffusion models can learn and reproduce an individual's identity from a few reference photos. This creates risks of unauthorized and misleading content generation.
The SEAL framework uses sparse adversarial protection to defend facial privacy. It modifies only 5% of the spatial locations in a facial image.
SEAL uses a differentiable top-k masking scheme to optimize perturbation locations and values. It uses a variance-oriented objective over the frozen VAE encoder's latent distribution to disrupt identity-consistent representations.
A saliency-guided dispersion penalty discourages perturbation concentration in perceptually salient facial regions. This helps preserve high visual quality.
Experiments on CelebA-HQ and VGGFace2 show SEAL achieves a PSNR exceeding 41 dB. It also reaches 0.97 SSIM and at most 0.07 LPIPS.
The method reduces identity similarity to at most 0.15 after diffusion personalization. This achieves strong identity protection while preserving high visual quality.
Key facts
- SEAL modifies only 5% of the spatial locations in a facial image.
- The method reduces identity similarity to at most 0.15 after diffusion personalization.
- SEAL achieves a PSNR exceeding 41 dB with 0.97 SSIM and at most 0.07 LPIPS.