Colloquium Archives for Academic Year 2021

Deep Implicit Coordination Graphs for Multi-agent Reinforcement Learning.

When: Wed, October 21, 2020 - 1:30pm
Speaker: Jayesh Gupta (Stanford and Microsoft Research) -
Abstract: Multi-agent reinforcement learning (MARL) requires coordination to efficiently solve certain tasks. Fully centralized control is often infeasible in such domains due to the size of joint action spaces. Coordination graph based formalization allows reasoning about the joint action based on the structure of interactions. However, they often require domain expertise in their design. In this talk, we will discuss the recently introduced deep implicit coordination graph (DICG) architecture for such scenarios. DICG consists of a module for inferring the dynamic coordination graph structure which is then used by a graph neural network based module to learn to implicitly reason about the joint actions or values. DICG allows learning the tradeoff between full centralization and decentralization via standard actor-critic methods to significantly improve coordination for domains with large numbers of agents.

Gap problems in higher dimensions: from Kronecker sequences to quantum oscillators

When: Wed, November 11, 2020 - 3:15pm
Where: Recording:
Speaker: Jens Marklof (University of Bristol) -

Abstract: Take a point on the unit circle and rotate it N times by a
fixed angle. The N points thus generated partition the circle into N
intervals. A beautiful fact, first conjectured by Hugo Steinhaus in
the 1950s and proved independently by Vera Sos, Janos Suranyi and
Stanislaw Swierczkowski, is that for any choice of N, no matter how
large, these intervals can have at most three distinct lengths. In
this lecture I will explore an interpretation of the three gap theorem
in terms of the space of Euclidean lattices, which will produce
various new results in higher dimensions, including nearest neighbour
distances in multi-dimensional Kronecker sequences, free flights in
the Lorentz gas, and quantum spectra of harmonic oscillators. The
lecture is based on joint work with Alan Haynes (Houston) and Andreas
Strombergsson (Uppsala). Recording

J. Marklof and A. Strombergsson, The three gap theorem and the space
of lattices, American Mathematical Monthly 124 (2017) 741-745
A. Haynes and J. Marklof, Higher dimensional Steinhaus and Slater
problems via homogeneous dynamics, Annales scientifiques de l'Ecole
normale superieure 53 (2020) 537-557
A. Haynes and J. Marklof, A five distance theorem for Kronecker
sequences, preprint arXiv:2009.08444

(Join Zoom Meeting

Meeting ID: 775 296 1556
Passcode: Bristol)

A Closer Look at Deep Policy Gradient Algorithms

When: Wed, November 18, 2020 - 1:00pm
Speaker: Logan Engstrom (Massachusetts Institute of Technology) -
Abstract: Deep reinforcement learning methods are behind some of the most publicized recent results in machine learning. In spite of these successes, however, deep RL methods face a number of systemic issues: brittleness to small changes in hyperparameters, high reward variance across runs, and sensitivity to seemingly small algorithmic changes. In this talk we take a closer look at the potential root of these issues. Specifically, we study how the policy gradient primitives underlying popular deep RL algorithms reflect the principles informing their development.

Multi-Agent Reinforcement Learning: Systems for Evaluation

When: Wed, January 6, 2021 - 1:00pm
Speaker: Justin Terry (University of Maryland) -
Abstract: Multi-Agent Reinforcement Learning: Systems for Evaluation

This talk discusses 4 papers and an ongoing project that deal with various aspects of designing software systems for multi-agent reinforcement learning, that allow for more productive and reproducible research, a democratization of research multi-agent reinforcement learning to university level researchers, and a new problems that are challenging to reinforcement learning in important ways. All works are centered around the PettingZoo project. This talk is specifically tailored to a very broad audience.


Diversity is All You Need: Learning Skills without a Reward Function

When: Wed, January 27, 2021 - 1:00pm
Speaker: Ben Eysenbach, Julian Abhishek (Carnegie Mellon University, Google Brain) -
Abstract: Intelligent creatures can explore their environments and learn useful skills without supervision. In this talk, we will present a method, 'Diversity is All You Need' (DIAYN), for learning useful skills without a reward function. We show how pretrained skills can provide a good parameter initialization for downstream tasks, and can be composed hierarchically to solve complex, sparse reward tasks. We will then discuss a close connection between autonomous skill discovery and meta-learning. Whereas typical meta-reinforcement learning algorithms require a manually-designed family of reward functions, we show how to use DIAYN to propose tasks for meta-learning in an unsupervised manner, effectively resulting in an unsupervised meta-learning algorithm. While there has been considerable work in this area in the past few years, a number of algorithmic and theoretical questions remain open. We plan to highlight some of these challenges at the end.

Waves: Building blocks in nature and in mathematics (AWM Distinguished Colloquium)

When: Wed, February 17, 2021 - 3:00pm
Speaker: Gigliola Staffilani (MIT) -
Abstract: In this talk I will first give a few examples of wave phenomena in nature. Then I will explain how, in order to understand these phenomena, mathematicians use tools from many different areas of mathematics, such as Fourier analysis, harmonic analysis, dynamical systems, number theory, and probability. I will also give examples of the beautiful interaction between the “concrete" and the “abstract,” and how these interactions constantly advance the boundaries of research.

Continuity of universally measurable homomorphisms

When: Wed, February 24, 2021 - 3:15pm
Speaker: Christian Rosendal (University of Illinois, Chicago) -
Abstract: Numerous results in Analysis concern rigidity of homomorphisms between topological groups or Banach algebras. In this talk, I will concentrate on topological rigidity of homomorphisms between topological groups, namely automatic continuity. Classical results of Banach, Sierpinski, Steinhaus and Weil established that every Haar measurable homomorphism between locally compact second countable groups is continuous and a similar result was found for Baire category by Pettis. In the late 1960s, Christensen introduced an appropriate notion of Haar null sets in Polish topological groups, which later found use in the theory of dynamical systems by influential work of Hunt, Sauer and Yorke. In connection with this work, Christensen asked whether every universally measurable homomorphism between Polish groups is continuous and gave positive answers for special cases. We will present the general solution to Christensen's problem and also show how the associated techniques have consequences in mathematical logic.

RIIT: Rethinking the Importance of Implementation Tricks in Multi-Agent Reinforcement Learning

When: Fri, February 26, 2021 - 10:00am
Speaker: Jian Hu, Seth Austin Harding (National Taiwan University, Taipei) -
Abstract: In recent years, Multi-Agent Deep Reinforcement Learning (MADRL) has been successfully applied to various complex scenarios such as playing computer games and coordinating robot swarms. In this talk, we investigate the impact of “implementation tricks” for SOTA cooperative MADRL algorithms, such as QMIX, and provide some suggestions for tuning. In investigating implementation settings and how they affect fairness in MADRL experiments, we found some conclusions contrary to the previous work; we discuss how QMIX’s monotonicity condition is critical for cooperative tasks. Finally, we propose the new policy-based algorithm RIIT that achieves SOTA among policy-based algorithms.

Deep Reinforcement Learning for Real-World Robotics

When: Tue, March 2, 2021 - 3:00pm
Speaker: Ilya Kuzovkin ( -
Abstract: OffWorld is developing a new generation of autonomous industrial robots to do the heavy lifting first on Earth, then on Moon, Mars and asteroids. We see reinforcement learning as one of major candidate technologies that could allow us to reach a high level of autonomy. While RL has achieved remarkable results in games and simulators, its adoption for real physical robots has been slow. In this talk we will go over a few projects we did at OffWorld that relate to applying RL on real robots, we then make the case that there is an apparent gap between RL community's aspirations to apply RL on real physical agents and its reluctance to move beyond simulators. To bridge this gap we introduce OffWorld Gym — a free access real physical environment and an open-source library that allows anyone to deploy their algorithms on a real robot using the familiar OpenAI gym ecosystem and without the burden of managing a real hardware system nor any knowledge of robotics.

Autonomous Navigation of Atratospheric Balloons Using Reinforcement Learning and The History of Atari Games in Reinforcement Learning

When: Wed, March 17, 2021 - 3:00pm
Speaker: Marc Bellemare (Google Brain, MILA, McGill University) -
Abstract: Marc Bellemare created the Arcade Learning Environment (how RL interfaces with Atari games) and using it co-created deep reinforcement while at DeepMind. He'll be giving a talk on his recent Nature paper on controlling Loon balloons using RL, as well as the history of Atari games in reinforcement learning.

Matchings as integer programs for kidney paired donation (kidney exchange) (AWM Distinguished Colloquium)

When: Wed, March 24, 2021 - 3:00pm
Speaker: Sommer Gentry (USNA) -

Abstract: People who volunteer as living kidney donors are often incompatible with their intended recipients. Kidney paired donation matches one patient and his or her incompatible donor with another pair in the same situation for an exchange. Let patient-donor pairs be the vertices of an undirected graph G, with edges connecting reciprocally compatible vertices. A matching in G is a feasible set of paired donations. Because the lifespan of a transplant depends on the immunologic concordance of donor and recipient, we weight the edges of G and seek a maximum edge-weight matching. I will first review various exponential-sized and polynomial-sized integer programming formulations proposed for this problem. Unfortunately, a maximum edge-weight matching might not have the maximum cardinality; there is a risk of an unpredictable trade-off between quality and quantity of paired donations. I will show that the number of paired donations is within a multiplicative factor of the maximum possible donations, where the factor depends on the edge weighting. Then I propose an edge weighting of G which guarantees that every matching with maximum weight also has maximum cardinality, and also maximizes the number of transplants for an exceptional subset of recipients, while favoring immunologic concordance. This result generalizes partly to k-way exchange and chains.

Slow chaos in flows on surfaces (Special Colloquium part of Dynamical systems Conference)

When: Fri, April 9, 2021 - 3:15pm
Speaker: Corinna Ulcigrai (University of Zurich) -
Abstract: Flows on surfaces describe many systems of physical origin and are one of the most fundamental examples of dynamical systems, studied since Poincaré. These systems often display a 'slow' form of butterfly effect, which makes them an important model of 'slowly chaotic' behaviour.
In the last decade, there have been a lot of advances in our understanding of the fine chaotic properties of smooth area-preserving flows on higher genus surfaces. During the talk, we will survey some of these properties and results and hint at some of the geometric and arithmetic mechanisms which explain them.

Dynamics in dimensions 1 and 3 (AWM Colloquium)

When: Wed, April 21, 2021 - 3:00pm
Speaker: Kathryn Mann (Cornell University)
Abstract: Suppose you have a group of transformations of a space. If you know something about individual transformations, can you extrapolate to say something global about the whole system? The paradigm example of this is an old theorem of Hölder: if you have a group of homeomorphisms of the real line and none of them fixes a point, then the group is abelian and the whole system is conjugate to an action by translations. My talk will be an illustrated introduction to this family of problems, including some recent joint work with Thomas Barthelmé that gives a new such result about groups acting on the line. As an application, we use this to prove rigidity results for a different, fascinating family of dynamical systems, Anosov flows in dimension 3.

Proving theorems with computers

When: Wed, April 28, 2021 - 3:15pm
Where: Online
Speaker: Kevin Buzzard (Imperial College, London, UK) -
Abstract: For decades now, computers have helped mathematicians to do large computations, via computer algebra software for example. Computers have also been able to help mathematicians to prove theorems, via computer proof software, which has also existed for decades. Whilst humans have been quite happy to pass on the job of doing tedious numerical calculations to computers, we have been far less keen to pass on the job of theorem proving to computers, because this is the fun part. However I believe it is only a matter of time before humans and computers start collaborating to prove theorems, and of course ultimately they might well get better than us at it. I will tell the story of how the Lean theorem prover went from the axioms of mathematics to checking deep theorems of Clausen and Scholze in a four year period, with a lot of human help. Let me stress that no background in computer proof systems or Scholze's mathematics will be assumed at all -- this is a talk for a general mathematical audience and would be suitable for undergraduates.