Language Models (LMs) are known to unintentionally memorize sensitive information from their training datasets, which can later be leaked during inference. To prevent such privacy breaches, we need reliable auditing and privacy-preserving mechanisms.
While various auditing methods are often costly and reactive, they only detect vulnerabilities after memorization has occurred. In this paper, we introduce a new method predicting which samples are likely to be memorized before memorization arises in a classification setting. Our approach is lightweight, doesn’t require any shadow model, and leverages information-theoretic signals between the model and its training data. Our promising results open the door to more proactive and targeted defenses for privacy in LLMs. This research was conducted as part of the "Responsible and Trustworthy AI" collaboration between École Polytechnique and Groupe Crédit Agricole, in collaboration with Sonia Vanier (Professor at DIX - Lead of the ORAILIX team), Jérémie Dentan (PhD candidate in ML security and privacy at L’X), (Davide Buscaldi (Lecturer at L’X) and Aymen Shabou (CTO & Head of AI - Datalab Groupe & AI Factory Group). Find out more on this topic via the following link: Predicting memorization within Large Language Models fine-tuned for classification Further details will be shared soon, during ECAI 2025 this October in Bologna.