Datasets and High-Fidelity Models for In-Cabin Behavior Understanding

Principal Investigator(s):

Hyun Soo Park, Assistant Professor, Computer Science and Engineering

Project summary:

The overarching goal of this project is to enable the measurement, analysis and prediction of in-car human behaviors. To achieve this goal, we propose to create a data acquisition system for the collection of large-scale in-cabin video datasets, using a multiview camera system that can allow the capture of realistic in-situ human behavior at multiple resolutions including 1) close up facial expression, head pose and eye gaze, 2) hand and arm movements and gestures, and 3) upper body movements and interaction. With this dataset, we will develop robust and generalizable algorithms for the 4D spatiotemporal representation of behaviors. To overcome the effects of dataset bias, we will design a new synthetic dataset augmentation scheme to generate diverse samples for domain adaptation to handle different vehicles, passenger occlusions, and novel scenarios.

Project details:

  • Project number: 2021041
  • Start date: 04/2021
  • Project status: Active
  • Research area: Transportation Safety and Traffic Flow