Using dummy data to protect privacy in data batches

A new mechanism uses dummy points to hide batch sizes and maintain privacy guarantees during data truncation.

Researchers developed a mechanism that uses a random number of dummy points. This method privatizes batch sizes and maintains privacy guarantees even when batches are truncated.

In Differentially Private Stochastic Gradient Descent (DP-SGD), servers process data in batches. Randomly sampling these batches allows for better privacy bounds through privacy amplification results.

Many sampling schemes like Poisson sampling and random allocation result in variable batch sizes. The privacy guarantee of these schemes requires that batch sizes are not revealed.

In distributed settings, a server or party observing communication traffic may learn these batch sizes. Memory and computation constraints may also require each batch to be a maximum size ss.

Truncating a batch when it exceeds the limit ss can result in a significant increase in privacy loss. This new mechanism uses dummy points to address these challenges.

The mechanism is easy to implement in a distributed setting. For Poisson sampling with truncated batches, it achieves better privacy--efficiency tradeoffs than the direct accounting for truncation given by (Ganesh, 2025).

Key facts

  • The mechanism uses a random number of dummy points to privatize batch sizes.
  • Truncation of batches to a maximum size ss can significantly increase privacy loss.
  • The mechanism works for both Poisson sampling and random allocation.
  • The expected number of dummy points is asymptotically smaller than what is needed to directly privatize batch sizes.

All sections

Edition