Federated Unlearning

The retained example rewinds model checkpoints and retrains after a client's deletion request. During retraining its aggregation strategy filters stale updates, while its client lifecycle removes the requested local samples.


cd examples/unlearning/fedunlearning
uv run fedunlearning.py -c fedunlearning_MNIST_lenet5.toml

This configuration uses Adam for ordinary retraining. The former AdaHessian recipe and its environment are preserved in the AdaHessian archive; the Adam recipe does not reproduce that optimizer variant.

Reference: Yi Liu, Lei Xu, Xingliang Yuan, Cong Wang, Bo Li. "The Right to be Forgotten in Federated Learning: An Efficient Realization with Rapid Retraining," INFOCOM, 2022.

Knot

Knot is implemented in examples/unlearning, which clusters the clients, and the server aggregation is carried out within each cluster only. Knot is designed under asynchronous mode, and unlearned by retraining from scratch in cluster. The global model will be aggregated at the end of the retraining process. Knot supports a wide range of tasks, including image classification and language tasks.


cd examples/unlearning/knot
uv run knot.py -c knot_cifar10_resnet18.toml
uv run knot.py -c knot_mnist_lenet5.toml
uv run knot.py -c knot_purchase.toml

Running on macOS

This example depends on cvxopt, which requires several system-level libraries for compilation:


brew install suite-sparse gsl glpk fftw

If you run into the following error:


src/C/umfpack.c:23:10: fatal error: 'umfpack.h' file not found

This is because the UMFPACK headers from SuiteSparse cannot be found. Set the proper environment variables before running:


export CVXOPT_BUILD_GLPK=0
export CVXOPT_BUILD_FFTW=0
export CVXOPT_BUILD_GSL=0
export CVXOPT_BUILD_DSDP=0
export CVXOPT_SUITESPARSE_LIB_DIR=/opt/homebrew/opt/suite-sparse/lib
export CVXOPT_SUITESPARSE_INC_DIR=/opt/homebrew/opt/suite-sparse/include/suitesparse

Important

Make sure you have a valid Mosek license before proceeding, as the Mosek optimization solver has been used.

Reference: N. Su, B. Li. "Asynchronous Federated Unlearning," in Proc. IEEE International Conference on Computer Communications (INFOCOM 2023).

Alignment with the paper

The Knot server extends the baseline retraining workflow but aggregates updates cluster-by-cluster (examples/unlearning/knot/knot_server.py:178-214), which is the cornerstone of Su and Li (INFOCOM 2023): each cluster trains independently and only its members' deltas are fused.

When do_optimized_clustering is enabled, the implementation gathers per-client training times and cosine similarities (examples/unlearning/knot/knot_server.py:660-811) and feeds them to the CVXOPT solver in examples/unlearning/knot/solver.py:1-118, matching the optimization-based client assignment proposed in Section 4 of the paper.

The clustered tester and rollback hooks (examples/unlearning/knot/knot_server.py:216-340 and examples/unlearning/knot/knot_trainer.py:13-84) ensure that each cluster is retrained in isolation until convergence before the global model is recombined, just as the paper specifies for asynchronous unlearning.