Augustin Godinot

Postdoctoral researcher, MPI-SWS

Robust ML Auditing using Prior Knowledge

Jade Garcia Bourrée*1,2, Augustin Godinot*2,1,3,4, Martijn de Vos5, Milos Vujasinovic5, Sayan Biswas5, Gilles Tredan6,7, Erwan Le Merrer1, Anne-Marie Kermarrec5

1Inria 2Université de Rennes 3IRISA/CNRS 4PEReN 5EPFL 6LAAS 7CNRS

ICML25arXivpostercodescholar

Do you remember Dieselgate? The car computer would detect when it was on a test-bench and reduce the engine power to fake environmental compliance. Well, this can happen with AI regulation too.

Interactions between the platform, the auditor and the users
Figure 1: Interactions between the platform, the auditor and the users

An audit is pretty straightforward.

  1. I, the auditor 🕵️ come up with questions to ask your model.
  2. You, the platform 😈 answer my questions.
  3. I look at your answers and decide whether your system abides by the law by computing a series of aggregate metrics.

Now, you know the metric, you know the questions, and I don’t have access to your model. Thus, nothing prevents you from manipulating the answers of your model to pass the audit. And this is very easy! In fact, any fairness mitigation method can be transformed into an audit manipulation attack.

There are two main approaches to avoid manipulations.

  • 🔒Crypto guarantees: the model provider is forced to commit their model and sign every answer.
  • 📐Clever ML tricks: the auditor uses information about the model (training data, model structure, …) to understand what is a “good answer”.

In this paper, we formalize the second approach as a search for efficient “audit priors”. We instantiate our framework with a simple idea: just look at the accuracy of the platform’s answers. Our experiments show that this can help reduce the amount of unfairness a platform could hide.

The amount of unfairness a platform can hide as a function of the auditor query budget
Figure 2: The concealable unfairness for different audit budgets (i.e., data samples from the labeled dataset). We highlight this for two features of the CelebA dataset (left) and for two different ML models trained on the ACSEmployment dataset (right).

If you want to read more about this, I encourage you to read the paper, but not only! Recently, there has been a lot of exciting works on robust audits, here are a few I enjoyed: