Augustin Godinot

Postdoctoral researcher, MPI-SWS

Manipulation-Proof Oblivious Audits against Deceptive Model Providers

Augustin Godinot1,2, Sofiane Azogagh3, Julien Ferry4, Sébastien Gambs5

1Inria 2Université de Rennes 3EURECOM 4Polytechnique Montréal 5Université du Québec à Montréal

AIES26postercodescholar
A provider patching a known audit set versus a hidden one
Figure 1: When the provider knows the audit set 𝑆, one flip suffices. When 𝑆 is hidden in 𝐶, it must fix all of 𝐶, and is twice as likely to hit a canary 🐦.

Under the DSA, regulators can audit the AI systems of large platforms. But the platform knows when it is audited, and on which questions. Showing one model to the regulator and another one to the users is then very easy: to hide unfairness, flipping a handful of answers on the audit set is enough. Hiding a demographic parity gap of 0.01 on 400 balanced queries takes 2 flips. This is the same manipulation problem as in our SaTML24 and ICML25 papers, but this time, we tackle the problem with a different approach.

Existing defenses either

  • 👀 look at the model differently (research APIs, monitoring, sock puppets), but still trust the provider;
  • 📚 bring external information (labeled data, priors, see our ICML25 paper), which is expensive for the auditor;
  • 🔒 verify every inference (zero-knowledge proofs, MPC, secure hardware), which is expensive for the provider and does not work for every model.
Probability of detecting the manipulation versus the number of canaries
Figure 2: Probability of catching a platform hiding half of its unfairness, for R²esPIR (blue) and a vanilla audit (grey).

In this paper, we mix canaries with lightweight cryptography. The intuition: faking fairness on a few known inputs is easy, faking it on a large set of inputs without getting caught is much harder.

  1. You, the platform 😈, label a large candidate set and commit to your answers.
  2. I, the auditor 🕵️, secretly read the answers of a small audit set with Private Information Retrieval: you never learn which ones.
  3. Among them are a few canaries 🐦, whose honest answers I got from your public service. If an answer differs, you are caught.

Not knowing what is audited, the platform has to fix the whole candidate set. On our datasets, it must flip about 4× more answers, which roughly doubles its chances of getting caught. And this costs 28 ms per query, without touching the model.

Want to know more? Have a look at the paper, the poster or the code.