Building AdaptiForget: Fixing What's Broken

Watching the SOTA methods fail over time forced me to engineer my own variant. I call it AdaptiForget.

The core failure of methods like MSG-KD under iterative deletion is that they use static parameters. I introduced three key mechanisms to fix this:

1. Periodic Mask Refreshes: We recalculate the saliency mask every 100 steps so we don’t rely on stale gradients. The network changes as it unlearns, and our targeting must change with it.
2. Adaptive Lambda Schedule: A dynamic KL-divergence weight that ramps up over time, allowing for aggressive forgetting early on, followed by strong utility protection later.
3. Dual Early-Stopping: A mechanism to halt the unlearning process the second we detect utility collapse or hit our target forget-advantage.

It actually works. It stays remarkably stable across 20 iterations without destroying the model’s performance on retained data.

But just as I was celebrating the success of AdaptiForget, a fatal flaw in my underlying dataset architecture hit me like a ton of bricks.