Deep Learning: Recent Advances and New Challenges


Russ Salakhutdinov

UPMC Professor of Computer Science in the Department of Machine Learning at CMU  
Director of AI research at Apple


Monday, September 30 11:00am - 12:00pm
Room 1202, EBU-3B

Faculty Host
Kamalika Chaudhuri

Carnegie Mellon University


I will first introduce XLNet, a generalized autoregressive pertaining method for natural language understanding that (1) enables learning bidirectional contexts by maximizing the expected likelihood over all permutations of the factorization order and (2) overcomes some limitations of BERT due to its autoregressive formulation. I will further show how XLNet integrates ideas from Transformer-XL, the state-of-the-art autoregressive model, into pertaining and demonstrate that XLNet outperforms BERT on a number of NLP tasks. In the second part of the talk, I will introduce the notion of structured memory as being a crucial part of an intelligent agent’s ability to plan and reason in partially observable environments. I will present a modular hierarchical reinforcement learning agent that can learn to store arbitrary information about the environment over long time lags, perform tasks specified by natural language instructions, and learn to build the map of the environment while generalizing across domains and tasks.


Russ Salakhutdinov is a UPMC Professor of Computer Science in the Department of Machine Learning at CMU and is a director of AI research at Apple. He received his PhD in computer science from the University of Toronto in 2009. After spending two post-doctoral years at the Massachusetts Institute of Technology Artificial Intelligence Lab, he joined the University of Toronto as an Assistant Professor in the Departments of Statistics and Computer Science. In 2016 he joined CMU. His primary interests lie in deep learning, machine learning, and large-scale optimization. He is an action editor of the Journal of Machine Learning Research, served as a program co-chair for ICML2019, served on the senior program committee of several top-tier learning conferences including NeurIPS and ICML, and was a program co-chair for ICML 2019. He is an Alfred P. Sloan Research Fellow, Microsoft Research Faculty Fellow, Canada Research Chair in Statistical Machine Learning, a recipient of the Early Researcher Award, Google Faculty Award, Nvidia's Pioneers of AI award, and is a Senior Fellow of the Canadian Institute for Advanced Research.