We aim to rigorously characterize the sensitivity of vision-in-the-loop driving controllers in increasingly complex visual tasks. While rooftop lidar provides a spectacular amount of high-rate geometric data about environment, there are a number of tasks in an autonomous driving system where camera-based vision will inevitably play a dominant role: dealing with lane markings and road signs, dealing with water/snow and other inclement weather conditions that can confuse a lidar, and even dealing with construction (orange cones), police officers, and pedestrians/animals. Furthermore, vision sensors are often fused with depth returns from a laser and other sensors as a part of the vehicle and obstacle estimation algorithms.
As an initial study, we will investigate the performance of a simple perception algorithm for lane detection and a simple controller for lane following, given visual scenes which capture some of the diversity of urban driving conditions here in Boston including complex on-road traffic markings at intersections and worn visual features. As the project progresses, we will attempt to simulate more and more of the visual world -- up to and including difficult volumetric effects such as fog or snow and dynamic obstacles such as pedestrians and other vehicles.
- M. Gharbi, J. Chen, J. Barron, S. Hasinoff, and F. Durand, “Deep Bilateral Learning for Real-Time Image Enhancement,” ACM Transactions on Graphics, vol. 36, no. 4, Jul. 2017 [Online]. Available: https://doi.org/10.1145/3072959.3073592, https://groups.csail.mit.edu/graphics/hdrnet/
- L. Anderson, T.-M. Li, J. Lehtinen, and F. Durand, “Aether: An Embedded Domain Specific Sampling Language for Monte Carlo Rendering,” ACM Transactions on Graphics, vol. 36, no. 4, Jul. 2017 [Online]. Available: https://doi.org/10.1145/3072959.3073704, https://people.csail.mit.edu/lukea/aether/