Hello,
We’d like to introduce a new tool for researchers and practitioners interested in Machine Unlearning (MU), the process of making a trained model “forget” specific data it was trained on. This capability is crucial for applications such as complying with data privacy regulations like the GDPR’s “right to be forgotten.”
However, evaluating MU methods is challenging. Retraining a model from scratch is highly inefficient, and it is difficult to systematically compare the numerous unlearning techniques available. An effective unlearning method must balance three key principles:
- Accuracy: Forgetting the specified data without degrading the model’s performance on the data it must retain.
- Efficiency: The amount of time and computation required to execute the unlearning process.
- Privacy: Ensuring the “forgotten” data leaves no discernible trace, thereby preventing its identification by an attacker (e.g., via a Membership Inference Attack).
To help explore these complex trade-offs, we developed the Unlearning Comparator, a web-based visual analytics system. It is designed to allow for the intuitive and systematic evaluation of different unlearning methods without writing complex code.
With this tool, you can:
- Visually compare how different unlearning methods (e.g., Fine-Tuning, Gradient Ascent, Random Labeling) alter a model’s performance and internal structure.
- Explore shifts in feature representations (embeddings) to understand how the model’s understanding of the data changes after unlearning.
- Simulate Membership Inference Attacks (MIAs) to verify if the specified data has been effectively “forgotten” from a privacy perspective.
Our goal is to make the evaluation of MU methods more systematic and accessible. This work is part of a research paper currently under review at IEEE Transactions on Visualization and Computer Graphics (TVCG).
The tool is available directly in your browser with no installation required.
- Live Demo: Machine Unlearning Comparator
- GitHub Repository: GitHub - gnueaj/Machine-Unlearning-Comparator: A Visual Analytics System for Comparative Evaluation of Machine Unlearning Methods
If you find the tool interesting or useful for your work, we would appreciate a star 
on GitHub. All feedback, questions, and suggestions are welcome.