Evaluation of Information-Theoretic Measuresin Echo State Networks on the Edge of Stability

4^th April 2019, 13:00
Misolov Torda
-

Abstract

Recent research has shown that an echo state network provides a viable model in neuroscience and computational science. The feedback nature of the recurrent layer makes optimal setting of the hyperparameters difficult and a subject of great importance. Previous work has demonstrated that tuning the network tooperate near the phase transition between ordered and disordered state leadsto the maximization of performance and of global information transfer within the reservoir. In this work we provide the theory and current state of the art estimation method for continuous random variables of two information measures,namely transfer entropy and active information storage, and we employ these measures to study the directed information transfer within the reservoir. We experiment with four datasets and show different behaviors of internal dynamics in relation to the stability of the network and the presented task. We also look at the complexity of neuron signals using Ragwitz–Kantz state space reconstruction method in various settings. The results provide interesting and novel insights into the computational properties of the echo state networks.