handling catastrophic forgetting for patch generation

[monperrus]

The R-Hero experiment has uncovered the existence of catastrophic forgetting problem for patch generation.

We will study how to overcome this problem.

(spinoff of #970 (comment))

[monperrus]

Early attempts to mitigate catastrophic forgetting typically consisted of memory systems that store previous data and that regularly replay old samples interleaved with samples drawn from the newdata (Robins 1993, 1995), and these methods are still used today (Gepperth & Karaoguz 2015, Rebuffi et al. 2016).
page 8

Continual Lifelong Learning with Neural Networks:A Review
https://arxiv.org/pdf/1802.07569.pdf

Advanced techniques:

• EWC from Overcoming catastrophic forgetting in neural networks https://arxiv.org/pdf/1612.00796.pdf
• Progress & compress: A scalable framework for continual learning. InICML, 2018 https://arxiv.org/pdf/1805.06370
• CLEAR: Experience Replay for Continual Learning 2019 http://papers.nips.cc/paper/8327-experience-replay-for-continual-learning.pdf

[monperrus]

Found by João Ferreira @jff:

• Serra, J., Suris, D., Miron, M. and Karatzoglou, A., 2018, July. Overcoming catastrophic forgetting with hard attention to the task. In International Conference on Machine Learning (pp. 4548-4557). PMLR.
  http://proceedings.mlr.press/v80/serra18a/serra18a.pdf

• Li, X., Zhou, Y., Wu, T., Socher, R. and Xiong, C., 2019, May. Learn to grow: A continual structure learning framework for overcoming catastrophic forgetting. In International Conference on Machine Learning (pp. 3925-3934). PMLR.
  http://proceedings.mlr.press/v97/li19m/li19m.pdf

[monperrus]

FYI,
Avalanche: an End-to-End Library for Continual Learning
https://github.com/ContinualAI/avalanche/