Selective Potentiality and Moving Focus for Interpreting Multi-Layered Neural Network

The present paper aims to demonstrate the existence of simplification forces in neural networks.These simplification forces can be represented by the simplest network, called "prototype."To extract the prototype, we need to identify necessary and important information during learning.The structural potentiality has been proposed to reduce information, aiming to reduce unnecessary information, but one of its problems lies in excessive information reduction.To preserve important information, we need to maximize or at least weaken the excessive information reduction.To solve this problem, we introduce a new potentiality called "selective potentiality," which allows us to move a focus field where a group of connection weights can be flexibly reduced.This method aims to replace the troublesome contradictory operations of potentiality reduction and augmentation with more concrete and manageable ones.The method was applied to an artificial dataset, in which linear and non-linear relations were introduced.The results confirmed that selective potentiality could be increased to weaken structural potentiality reduction.The selective potentiality showed strong forces of simplification throughout the entire learning process.By seeking the simplest prototype, additional results were obtained, where networks tried to infer the outputs, enhancing both linear and non-linear inputs for better generalization.