New way to learn data patterns while protecting individual privacy

Researchers developed efficient algorithms to estimate discrete distributions under person-level differential privacy.

Researchers studied how to learn a discrete distribution under person-level differential privacy in a batched setting. They want to estimate a distribution μ\mu over $[k]$ with a total-variation error α\alpha.

The study involves nn users. Each user provides mm i.i.d. samples from the unknown distribution μ\mu.

The algorithms use a new degree-6 sum-of-squares estimator. This estimator learns from untrusted batches. It uses a trace-bounded diagonal majorizer for the difference between empirical covariance and multinomial covariance.

The researchers used semidefinite-programming duality and a symmetric form of Grothendieck's inequality. These tools prove that clean data admit a specific certificate. This certificate preserves the mean-dependent variance of multinomial batch averages.

Under pure (ε,0)(\varepsilon,0)-DP, the algorithm achieves n=O~(k/(α2m)+k/(αmε)+k/ε)n = \widetilde O(k/(\alpha^2 m)+k/(\alpha\sqrt{m}\varepsilon)+k/\varepsilon). This matches the information-theoretic optimum up to logarithmic factors.

Under approximate (ε,δ)(\varepsilon,\delta)-DP, the algorithm achieves n=O~(k/(α2m)+k/(αmε)+k+log⁡(1/δ)/ε)n = \widetilde O(k/(\alpha^2 m)+k/(\alpha\sqrt{m}\varepsilon)+k+\log(1/\delta)/\varepsilon). A statistical query lower bound rules out a uniformly smaller order of error under fixed polynomial SQ resource bounds.

Key facts

  • The algorithms work under both pure and approximate differential privacy.
  • The pure differential privacy guarantee matches the information-theoretic optimum up to logarithmic factors.
  • A statistical query lower bound rules out a uniformly smaller order of error under fixed polynomial SQ resource bounds.

All sections

Edition