Tackling Sequential Entanglement in Split Unlearning

Split learning enables distributed model training by partitioning a neural network between clients and a server and exchanging information pertained to intermediate activations. Recent studies on machine unlearning emerged, where one aims to erase the impact of certain data points on the trained model. While existing related work focused on centralized and federated scenarios, the problem of split unlearning is much under-explored. To fill this gap, we propose a split unlearning algorithm that applies orthogonal projection to gradient caches on both the clients and the server to excise contributions from data slated for removal and then fine-tunes the model on the remaining data. A key technical challenge is associated with the model split and the sequential entanglement in split learning, where a client cache lacks entries for the data points to be removed from the other clients. We address this by employing a rounding mechanism that substitutes missing gradients with their nearest available approximations. We provide a convergence analysis of our algorithm, which achieves an O(1/T) rate under non-convex objectives. Numerical experiments show that our algorithm can effectively unlearn various data points while achieving an accuracy close to the retrained benchmarks.