View Abstract
Abstract: The “One Learning Algorithm Hypothesis” summarizes strong experimental evidence that the human brain processes visual, acoustic, and haptic signals, for perception and cognition with a workflow that corresponds to the same abstract architectural model. We present our most recent results on the development and performance evaluation of a universal machine learning architecture inspired by this hypothesis. The abstract architecture proposed is comprised of a multi-resolution processing front, followed by a feature extractor, followed by two “local” learning modules (first an unsupervised one, followed by a supervised one), followed by a deterministic annealing module. There are two global feedback loops, one to the multiresolution processor and one to the feature extractor. Innovative analytical methods and results include: multi-resolution hierarchy, use of Bregman divergences as dissimilarity measures, multi-scale stochastic approximation, multi-scale approximation to Bayes decision surfaces, optimization-information duality. We demonstrate the superior performance and characteristics of the resulting algorithms including: domain agnostic, on-line progressive learning, interpretability, robustness to noise and adversarial attacks, computable performance-complexity tradeoff. We present several applications in signal processing, graph problems, estimation and control.