Git Re-Basin: Merging Models modulo Permutation Symmetries

Really interesting paper about how we can merge models, even those trained on disparate datasets, with no impact on the loss! Lots of implications, not least of all for federated learning

2 Likes