Learning to Learn : Generalization of Evolved Plasticity Rules in Recurrent Neural Networks

In machine learning, one often looks towards biology to find mechanisms to replicate. One of these mechanisms we observe in biology is synaptic plasticity rules. Plasticity rules allow for an artificial neural network to learn new tasks. By changing synapse weights based on these rules, an agent can adapt itself. Choosing proper rules then becomes a new challenge. Previously, gradient descent has been used to optimize one such set of rules. We use an evolutionary algorithm, specifically evolution strategies, to learn rules for recurrent neural networks. These are algorithms based on biological evolution which do not require a measure of error and may thus be applicable to a wider set of problems. Our results show that this approach may have potential, obtaining better-than-chance performance on classification tasks. This results in a proof of concept that enables further research in this area.