UC Berkeley to lead $10M NSF/Simons Foundation program to investigate theoretical underpinnings of deep learning

5-year project includes 11 researchers at 8 institutions

Berkeley, Calif.--The National Science Foundation (NSF) and Simons Foundation today (Aug. 25) awarded $10 million to a UC Berkeley-led program to gain a theoretical understanding of deep learning, which is making significant impacts across industry, commerce, science, and society.

Although deep learning is a widely used artificial intelligence approach for teaching computers to learn from data, its theoretical foundations are poorly understood, a challenge that the project will address. Understanding the mechanisms that underpin the practical success of deep learning will allow researchers to address its limitations, including its sensitivity to data manipulation.

The five-year project will be led by Professor Peter Bartlett, with Professor Bin Yu as co-PI. Both have appointments in the university’s Departments of Statistics and of Electrical Engineering and Computer Sciences (EECS). Other participating institutions are Stanford University, the Massachusetts Institute of Technology, UC Irvine, UC San Diego, Toyota Technological Institute at Chicago, EPFL in Lausanne, Switzerland, and the Hebrew University in Jerusalem.

The NSF and the Simons Foundation Division of Mathematics and Physical Sciences are partnering to fund research in the Mathematical and Scientific Foundations of Deep Learning. Deep Learning is part of a broader family of machine learning methods based on artificial neural networks that digest large amounts of raw data inputs and train AI systems with limited human supervision. 

Most of the research and education activities will be hosted by the Simons Institute for the Theory of Computing, UC Berkeley's global center for collaboration in theoretical computer science, in the form of structured programs of varying themes.

“Our excitement over receiving this award is that we will be digging into the theoretical foundations of deep learning,” said Bartlett, who is also Associate Director of the Simons Institute for the Theory of Computing. “The recent success in machine learning has been driven by a spirit of craftsmanship by people who find ways to make this technology successful. But much of this work contradicts a lot of our classical understanding of statistical methodology, and there are many things we don’t understand about how and why these systems work.” 

Yu, who has been using deep learning to advance research in neuroscience said that the approach is also used to analyze medical images, study cardiovascular disease and improve precision medicine, and these high-stake applications call for an understanding of deep learning theoretically.

“This is really cutting-edge research at the frontier of data science theory and practice,” said Yu, who is also in Berkeley’s Division of Computing, Data Science, and Society (CDSS). “It’s leading us intellectually where we want to go with deep learning. Theory can help improve our practice, and that practice then drives theory.”

Yu said the project is a good fit for CDSS because although deep learning offers great potential for solving problems, it’s critical to understand how it works because it could cause harm if not used properly, especially in areas such as health and medicine.

CDSS fosters cross-campus partnerships, bringing together researchers in areas ranging from economics to social welfare, climate studies to public policy, computer science, electrical engineering, and statistics to biomedicine to apply tools like deep learning to solving societal problems. The new project builds on work by the EECS Department, which is jointly housed in CDSS and Berkeley’s College of Engineering, and the Statistics Department, which is in both CDSS and the Division of Mathematical and Physical Sciences.

Bartlett said that the Simons Institute, as a fully embedded research center within the university with its distinguished international network of scientists and scholars, will enhance the breadth and depth of the research. 

In addition to research, Berkeley’s Data Science Education Program under CDSS educates students in data sciences, giving them the tools to look at data confidently, critically, and ethically in whatever field they choose for a career. To contribute to that mission, the newly funded project will organize workshops open to the public, sponsor an annual summer school for grad students, postdocs and faculty, and train a diverse cohort of postdocs in the theoretical basis of deep learning.

“We’re very excited that we’re one of two groups to receive an award,” Bartlett said. “I’m really pleased with the strength of our team; in our online meetings to develop the proposal, it was already clear that this is a group that’s going to work really well together.”

Other participants are Emmanuel Abbe, EPFL; Mikhail Belkin, UC San Diego; Amit Daniely, Hebrew University; Andrea Montanari, Stanford University; Elchanan Mossel, Alexander Rakhlin and Nike Sun, MIT; Nathan Srebro, Toyota Technological Institute at Chicago; and Roman Vershynin, UC Irvine.

The project will also benefit by gaining perspectives from two industry advisors; one from Google and the second from Mobileye, an Israeli subsidiary of Intel that develops self-driving vehicles and advanced driver-assistance systems.