Memory Consolidation with Orthogonal Gradients for avoiding Catastrophic Forgetting
Tamizharasan Kanagamani, Rupak Krishnamurthy, V. Srinivasa Chakravarthy, Balaraman Ravindran, Ramshekhar N. Menon
The paper proposes a regularization-based model to prevent catastrophic forgetting in deep networks by constraining parameter updates to directions orthogonal to previous task gradients, showing improved performance on standard datasets.
bioRxiv · Feb 25, 2022 · PreprintNeuroscience