By Josh Baxt
Machine learning models tend to need a lot of labeled data to produce accurate results. Computer vision analyzes digital images to figure out what’s going on in the picture but might need thousands of annotated images to learn the difference between dogs and cats.
That is hardly effective in the real world. Take self-driving cars. In an incredibly dynamic environment, how can machine learning help predict a sudden left turn or an erratic pedestrian? These systems must draw conclusions with much less data.
To help make this happen, UC San Diego Computer Science and Engineering (CSE) professor Rose Yu has received a $370,000 grant from the
Army Research Office to advance this research. And her project, Physics-Guided Learning for Sample Efficient Spatiotemporal Decision Making, is already bearing fruit: Two of her papers, co-authored with CSE Ph.D. student Rui Wang and postdoc Robin Walters, along with Northeastern graduate student Jinxi Li, were recently accepted by The Ninth International Conference on Learning Representations, the highest impact factor AI conference.
As part of her research, she is incorporating physics into machine learning models. Whether it’s understanding the dynamics of freeways, ocean currents or subatomic particles, this physics-based approach can give machine learning a more predictive punch.
“The key issue is we cannot afford such high sample complexity in our system, and we want to build machine learning models that can learn from just a few examples. We want to develop machines that can learn like babies,” said Yu.
"Machine Learning has a large role to play in future warfare and in protecting our soldiers; however, extant ML models are easily fooled by adversarial disturbances in inputs, rendering them useless,” said Purush Iyer, branch chief, network sciences, with the ARO, an element of U.S. Army Combat Capabilities Development Command’s Army Research Laboratory.
“For ML to be effective with incomplete data, which is most common in the real world, ML models that incorporate mechanistic physics-based constraints is a necessity, as is being developed at UC San Diego,” Iyer said.
Cars and Boats
In one paper, Trajectory Prediction using Equivariant Continuous Convolution, Yu and colleagues explore how fluid dynamics can inform efforts to understand vehicle and pedestrian trajectories.
“In a sense, modeling cars on the road is even harder than particle physics,” said Yu, “because we have to model the physics behind the trajectories in terms of velocity and acceleration, as well as human psychology.”
Yu hopes this work will give self-driving cars, and other vehicles, better tools to understand what’s happening around them, predict potential hazards and avoid accidents. Predicting pedestrian motion adds another piece to the puzzle, giving self-driving vehicles better “intuition” to move safely in a complex physical world.
The second paper, Incorporating Symmetry into Deep Dynamics Models for Improved Generalization, takes a similar approach to ocean currents, incorporating the laws of physics into deep learning models.
“Ocean currents are complicated because we don't have a good model to predict their evolution,” said Yu.
This research also supports efforts to study climate change, as turbulence and ocean currents play a significant role in climate science. The common thread in all this work is leveraging existing knowledge in physics and other disciplines to build machine learning models that can make better predictions with less real-world data.
“Because we have prior knowledge and domain expertise, we can quickly bootstrap into new domains,” said Yu. “The high-level idea behind these two papers is that we can build these types of intuition into machine learning models. They’re not starting from scratch but building on existing human knowledge so they can learn quickly from a few examples.”