View Abstract
Abstract: Deep Learning and Artificial Intelligence have attracted enormous attention recently. The race to design and manufacture âbrain-likeâ computers is on and several companies have produced various such chips. Yet, the current state of affairs is very unsatisfactory and ad hoc. We describe a mathematical framework we have developed that provides a hierarchical architecture for learning and cognition. The architecture combines a wavelet preprocessor, a group invariant feature extractor and a hierarchical (layered) learning algorithm. There are two feedback loops, one back from the learning output to the feature extractor and one all the way back to the wavelet preprocessor. We show that the scheme can incorporate all typical metric differences but also non-metric dissimilarity measures like Bregman divergences. The learning module incorporates two universal learning algorithms in their hierarchical tree-structured form, both due to Kohonen, Learning Vector Quantization (LVQ) for supervised learning and Self-Organizing Map (SOM) for unsupervised learning. We demonstrate the superior performance of the resulting algorithms and architecture on a variety of practical problems including: speaker and sound identification, simultaneous determination of sound direction of arrival speaker and vowel ID, face recognition. We demonstrate how the underlying mathematics can be used to provide systematic models for design, analysis and evaluation of deep neural networks. We describe current work and plans on mixed signal (digital and analog) micro-electronic implementations that mimic architectural abstractions of the cortex of higher-level animals and humans, for sound and vision perception and cognition. The resulting architecture is non-von Neumann (i.e. computing and memory are not separated in the hardware) and neuromorphic. We call the resulting chip class âCortex-on-a-Chip.â