Elman and Jordan Recurrence in Convolutional Neural Networks Using Attention Window

Retina of the human eyes constantly foveate on the target objects in a large visual field to recognize the object, and that foveated region is called central fovea. At each instance of the eye movements, human visual system integrates the significant information observed at the central fovea of the visual field inside a spatiotemporal memory to perform an object’s pattern recognition task. This integration property is widely useful in an artificial domain when the image size is too large. Therefore, presenting the full image to the visual pattern recognition systems in deep learning research is computationally expensive and biologically implausible. Being inspired from this biological hypothesis, we proposed five variations of Elman and Jordan recurrence in convolutional neural networks (EJRCNNs). Each of the five networks takes input of a series of small attention windows, which is cropped out from different locations in the image. Here, the attention window contributes as central fovea of the human visual system. The proposed networks integrate the information presented in all of the attention windows inside the context layers using recurrence connections in convolutional and fully connected layers. After processing all of the attention windows, networks perform the classification task. Elman and Jordan recurrences only in fully connected layers were partially explored in some previous studies using the full image. Each of the networks is trained on the MNIST [1] handwritten digit database. From our extensive experiments, the networks provide a better correlation of the spatiotemporal information by outperforming the RNN.