Zhi-Li Zhang, a professor in the U's Department of Computer Science and Engineering (CS&E) and a CTS Research Scholar, has received a $4.25 million five-year grant from the National Science Foundation for his work on autonomous vehicles (AVs).
The project, “Integrated Networking, Edge System and AI Support for Resilient and Safety-Critical Tele-Operations of Autonomous Vehicles,” aims to facilitate the safe and incremental adoption of tele-operated AVs to address societal challenges in transportation, while accelerating AV technology toward full autonomy.
Researchers and transportation companies around the world have been developing self-driving cars for a number of years. AVs use artificial intelligence (AI) and machine learning to identify common environments and situations in order to make driving decisions. Despite the advances over the years, there are still situations that AVs don't know how to handle. Zhang’s project leverages the advances in computing systems, like fifth-generation (5G) networks, to support “tele-operations” with a human operating AVs remotely when a complex situation arises.
“When more complex problem solving is required, the AV would need to send a large amount of rapidly changing data over the wireless network for the tele-operator to step in,” Zhang says. “Our work will explore ways to make that data transfer more reliable, secure, and safe. We are also working on improving the generative AI, so it can better predict when these complex situations may arise—like during cold weather or rain.”
AVs rely on the “vision” of their sensors to make driving decisions, making certain weather events problematic for the technology. With most AV trials and research being conducted in California and the South, where weather is more consistent, this project will aim to improve AV vision with the help of Midwest weather patterns.
Building off his work on shared autonomous vehicles (SAVs), Zhang hopes that augmenting AI with human intelligence is the missing piece that will make AVs safer and more reliable. These tele-operators won’t be controlling the car at all times, but they need to be ready to act quickly when a situation arises. One of the key challenges with this project is building systems that can keep tele-operators engaged at the right times, avoiding what Zhang calls “out of the loop syndrome.”
“This is interdisciplinary work,” says Zhang. “We need to understand the human side of things to make sure that the AVs and alert systems can communicate effectively with tele-operators. Our AI can be even more effective if we incorporate human intelligence in decision making."
Cybersecurity is another key aspect, Zhang explains. "These systems need to be extremely secure to ensure that only designated individuals can remotely operate vehicles. There are a number of systems within the AVs and traffic networks that are constantly sharing information, so security is a complex issue.”
CS&E associate professors Kangjie Lu and Catherine Zhao will lead work on network security and computer vision, respectively. CS&E lecturer and research assistant Eman Ramadan, as well as professors Rajesh Rajamani and Nichole Morris from the Department of Mechanical Engineering, are also key contributors.
In addition, Zhang will work closely with collaborators at the University of Michigan, including Morley Mao and Henry Liu, the Minnesota Department of Transportation, and a number of industry partners, such as Ericsson, InterDigital, Lear Corporation, ATT Labs, Microsoft, and Nokia.
“The ultimate goal is to build a platform that can facilitate tele-operated AVs in any scenario,” Zhang said. “The technology would be open source so anyone could use it and adapt it. In the future, we hope AVs can be used to address societal problems and improve the accessibility of transportation.”
Adapted from an article published by the Department of Computer Science and Engineering.
