RIT on Deep Learning Archives for Fall 2018 to Spring 2019
Organizational Meeting
When: Fri, September 1, 2017 - 12:00pmWhere: Kirwan Hall 1311
Speaker: Organizational Meeting () -
Introduction to Deep Learning
When: Fri, September 8, 2017 - 12:00pmWhere: Kirwan Hall 1311
Speaker: Ilya Kavalerov (UMD) -
Abstract: In this talk, I give an overview of the concepts and applications of deep neural networks. As the title suggests, this talk is introductory and requires no prior knowledge of the subject. Everyone is invited to attend!
Introduction to Convolutional Neural Networks
When: Fri, September 15, 2017 - 12:00pmWhere: Kirwan Hall 3206
Speaker: Shujie Kang (UMD) -
Abstract: This is a continuation of last week's talk. I introduce convolutional neural networks.
Training a Convolutional Neural Network
When: Fri, September 29, 2017 - 12:00pmWhere: Kirwan Hall 3206
Speaker: Shujie Kang (UMD) -
Abstract: I introduce commonly used algorithms to train a convolutional neural network.
A closer look at the ADAM optimizer
When: Fri, October 6, 2017 - 12:00pmWhere: Kirwan Hall 3206
Speaker: Eric Oden (UMD) -
Abstract: I build up on Shujie's talk by taking a closer look at the ADAM optimizer she mentioned towards the end of her talk.
Reinforcement learning and Trust Region Policy Optimization
When: Fri, October 13, 2017 - 12:00pmWhere: Kirwan Hall 3206
Speaker: Cheng Jie (UMD) -
Abstract: I give an overview of the mathematical framework of Reinforcement learning: Markov Decision Process. I will introduce basic policy optimization algorithms used to train the reinforcement learning model. Specifically, the talk will explore Trust Region Policy Optimization, a recently developed algorithm, widely used in Training Deep Reinforcement Learning.
Actor-Critic Method
When: Fri, October 20, 2017 - 12:00pmWhere: Kirwan Hall 3206
Speaker: Nathaniel Monsoon (UMD) -
Analysis of Convergence of Back-Propagation
When: Fri, October 27, 2017 - 12:00pmWhere: Kirwan Hall 3206
Speaker: Andrew Lauziere (UMD) -
Abstract: I will present the paper Efficient BackProp by Yann LeCun et al.
Batch Training of Neural Networks
When: Fri, November 10, 2017 - 12:00pmWhere: Kirwan Hall 3206
Speaker: Roozbeh Yousefzadeh (UMD (CS)) -
Abstract: I will present two recent papers on batch training of neural networks. The first paper (Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour) focuses on the size of mini-batches and the optimization difficulties that arise in distributed training. The second paper (Train longer, generalize better: closing the generalization gap in large batch training of neural networks) proposes an algorithm called "Ghost Batch Normalization".
A Proof of Convergence For The Stochastic Gradient Descent Method on Convex Cost Functions
When: Fri, November 17, 2017 - 12:00pmWhere: Kirwan Hall 3206
Speaker: Daniel Mourad (UMD) -
Optimization methods for discrete and saddle-point problems in machine learning
When: Fri, December 1, 2017 - 12:00pmWhere: Kirwan Hall 3206
Speaker: Tom Goldstein (UMD/CS) - https://www.cs.umd.edu/~tomg/
Abstract: We'll discuss two recent advanced in deep learning: adversarial neural networks, and quantized nets. Adversarial nets is a recent method for building generative models that requires the solution of complex saddle-point problems. We develop a simple prediction methods for stabilizing the training of saddle-point problems. Then, we'll discuss quantized networks, which use low-precision weights to compress and accelerate neural networks. We discuss the theory of quantized networks, and when/why they are trainable.
Mathematics in Machine Learning: Present and Future
When: Fri, December 8, 2017 - 12:00pmWhere: Kirwan Hall 3206
Speaker: Wojtek Czaja (UMD (math)) -
NeuroEvolution of Augmenting Topologies (NEAT): A Genetic Algorithm for Neural Networks
When: Wed, February 7, 2018 - 10:00amWhere: Kirwan Hall 3206
Speaker: Brandon Alexander (UMD) -
Training Quantized Nets: A Deeper Understanding
When: Thu, February 22, 2018 - 10:00amWhere: Kirwan Hall 3206
Speaker: Liam Fowl (UMD) -
Training Neural Networks Without Gradients: A Scalable ADMM Approach
When: Thu, March 1, 2018 - 10:00amWhere: Kirwan Hall 3206
Speaker: Zeyad Emam (UMD) -
Abstract: I will present the paper Training Neural Networks Without Gradients: A Scalable ADMM Approach by Prof Goldstein et al.
Spectral Networks and Locally Connected Networks on Graphs
When: Thu, March 8, 2018 - 10:00amWhere: Kirwan Hall 3206
Speaker: Addison Bohannon (UMD) -
Abstract: I will present the paper Spectral Networks and Locally Connected Networks on Graphs by Bruna et al.
FiLM: Visual Reasoning with a General Conditioning Layer
When: Thu, March 15, 2018 - 10:00amWhere: Kirwan Hall 3206
Speaker: Matthew Guay (NIH/NIBIB) -
Abstract: I will present on the paper named in the title, which has the abstract:
We introduce a general-purpose conditioning method for neu-
ral networks called FiLM: Feature-wise Linear Modulation.
FiLM layers influence neural network computation via a sim-
ple, feature-wise affine transformation based on conditioning
information. We show that FiLM layers are highly effective
for visual reasoning — answering image-related questions
which require a multi-step, high-level process — a task which
has proven difficult for standard deep learning methods that
do not explicitly model reasoning. Specifically, we show on
visual reasoning tasks that FiLM layers 1) halve state-of-the-
art error for the CLEVR benchmark, 2) modulate features in
a coherent manner, 3) are robust to ablations and architectural
modifications, and 4) generalize well to challenging, new data
from few examples or even zero-shot.
Mallat’s scattering transform, the Fourier scattering transform, and applications in hyperspectral imagery
When: Thu, March 29, 2018 - 10:00amWhere: Kirwan Hall 3206
Speaker: Ilya Kavarelov (UMD (ECE)) -
Transforming machine learning heuristics into provable algorithms: classical, stochastic, and neural
When: Thu, April 5, 2018 - 10:00amWhere: CHM 0115
Speaker: Cheng Tang (GWU) - https://sites.google.com/site/chengtanggwu/
Abstract: A recurring pattern in many areas of machine learning is the empirical success of a handful of "heuristics", i.e., any simple learning procedure favored by practitioners. Many of these heuristic techniques lack formal theoretical justification. For unsupervised learning, Lloyd's k-means algorithm, while provably exponentially slow in the worst-case, remains popular for clustering problems arising from different applications. For supervised learning, random forest is another example of a winning heuristic with many variants and applications. But the most prominent example is perhaps the blossoming field of deep learning, which is almost entirely composed of heuristics; the practical success of a deep learning algorithm usually relies on an experienced user skillfully and creatively combining heuristics. In this talk, I will discuss some of my thesis work in advancing the theoretical understanding of some of the most widely-used machine learning heuristics.
How non-convex are neural net loss functions, and what do they look like?
When: Thu, April 12, 2018 - 10:00amWhere: Kirwan Hall 3206
Speaker: Thomas Goldstein (UMD (UMIACS)) - https://www.cs.umd.edu/~tomg/
Abstract: This talk investigates the structure of neural network loss functions. Using a range of visualization methods, we explore the non-convex and convex structures present in loss functions, and how neural network architecture impacts these structures. I'll also discuss the implications this has on neural optimization, and present situations where bad actors can exploit neural loss functions to manipulate the behavior of classifiers.
Deep Learning - now and the future, an overview
When: Thu, April 19, 2018 - 10:00amWhere: Kirwan Hall 3206
Speaker: James Yorke (UMD ) - http://www.chaos.umd.edu/~yorke/
Accelerated Gradient Descent Escapes Saddle Points Faster than Gradient Descent
When: Thu, May 3, 2018 - 10:00amWhere: Kirwan Hall 3206
Speaker: Wojtek Czaja (UMD (MATH)) -
Abstract: I will present the paper Accelerated Gradient Descent Escapes Saddle Points Faster than Gradient Descent by Chi Jin, Praneeth Netrapalli, and Michael I. Jordan