A Parallel Autonomy System: Data-Driven and Model-Based Parallel Autonomy with Robustness and Safety Guarantees

Parallel Autonomy System
Daniela Rus, Sertac Karaman

This project will develop a parallel autonomy system to create a collision-proof car. We will instrument a Toyota vehicle with a suite of sensors pointed at the environment and at the driver to create situational awareness, both inside and outside the vehicle. We will develop and integrate the perception and decision- making software components to implement the parallel autonomy software core. We will also develop and implement novel algorithms that take control of the vehicle in dangerous situations to prevent accidents. We will evaluate the system both in simulation and in testing on a closed-course near MIT. We will increase the difficulty of our tests over time, moving from lower to higher speeds and from low to high environmental complexity. We will improve the system over time, following a modular design in which we continually update critical system components with improved algorithms for perception, motion planning and control.

This is a continuation of the project "A Parallel Autonomous Driving System" by Daniela Rus, John Leonard, Sertac Karaman.


  • A. Amini, I. Gilitschenski, J. Phillips, J. Moseyko, R. Banerjee, S. Karaman, D. Rus. "Learning Robust Control Policies for End-to-End Autonomous Driving From Data-Driven Simulation". IEEE Robotics and Automation Letters [Online]. Available: https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=8957584

  • M. Abu-Khalaf, S. Karaman, and D. Rus, “Shared Linear Quadratic Regulation Control: A Reinforcement Learning Approach,” in IEEE Conference on Decision and Control (accepted), 2019 [Online]. Available: https://arxiv.org/abs/1905.11524

  • F. M. Naser, I. Gilitschenski, A. Amini, L. Christina, G. Rosman, S. Karaman, and D. Rus, “Infrastructure-Free NLoS Obstacle Detection for Autonomous Cars,” in IROS 2019 (Accepted), 2019.

  • X. Du, M. H. A. Jr, S. Karaman, and D. Rus, “A Unified Pipeline for 3D Detection and Velocity Estimation of Vehicles,” in ISRR 2019 (Accepted)), 2019.

  • S. Gumussoy and M. Abu-Khalaf, “Analytic Solution of a Delay Differential Equation Arising in Cost Functionals for Systems with Distributed Delays,” IEEE Transactions on Automatic Control, Jun. 2019 [Online]. Available: https://doi.org/10.1109/TAC.2019.2921658
  • B. Araki, I. Gilitschenski, T. Ogata, A. Wallar, W. Schwarting, Z. Choudhury, S. Karaman, and D. Rus, “Range-based Cooperative Localization with Nonlinear Observability Analysis,” in The 22nd Intelligent Transportation Systems Conference (ITSC2019), 2019 [Online]. Available: https://www.itsc2019.org

  • A. Amini, G. Rosman, S. Karaman, and D. Rus, “Variational End-to-End Navigation and Localization,” in 2019 International Conference on Robotics and Automation (ICRA), 2019 [Online]. Available: https://doi.org/10.1109/ICRA.2019.8793579

  • F. M. Naser, “Detection of Dynamic Obstacles out of the Line of Sight for Autonomous Vehicles to increase Safety based on Shadows,” 2019 [Online]. Available: https://dspace.mit.edu/handle/1721.1/121657

  • F. Naser, I. Gilitschenski, G. Rosman, A. Amini, F. Durand, A. Torralba, G. W. Wornell, W. T. Freeman, S. Karaman, and D. Rus, “ShadowCam: Real-Time Detection of Moving Obstacles Behind A Corner For Autonomous Vehicles,” in 21st IEEE International Conference on Intelligent Transportation Systems (ITSC 2018), 2018 [Online]. Available: https://doi.org/10.1109/ITSC.2018.8569569

  • L. Liebenwein, C. Baykal, I. Gilitschenski, S. Karaman, and D. Rus, “Sampling-Based Approximation Algorithms for Reachability Analysis with Provable Guarantees,” in Robotics: Science and Systems, 2018 [Online]. Available: http://www.roboticsproceedings.org/rss14/p14.pdf

  • A. Amini, L. Paull, Thomas Balch, Sertac Karaman, and D. Rus, “Learning Steering Bounds for Parallel Autonomous Systems,” in ICRA 2018, 2018 [Online]. Available: https://doi.org/10.1109/ICRA.2018.8461253

  • A. Amini, W. Schwarting, G. Rosman, B. Araki, S. Karaman, and D. Rus, “Variational Autoencoder for End-to-End Control of Autonomous Driving with Novelty Detection and Training De-biasing,” in IROS 2018, 2018 [Online]. Available: https://doi.org/10.1109/IROS.2018.8594386

  • A. Amini, A. Soleimany, S. Karaman, and D. Rus, “Spatial Uncertainty Sampling for End to End Control,” in Bayesian Deep Learning, NIPS 2017 Workshop, Long Beach, CA, US, 2017 [Online]. Available: http://bayesiandeeplearning.org/2017/papers/25.pdf

  • W. Schwarting, J. Alonso-Mora, L. Paull, S. Karaman, and D. Rus, “Safe Nonlinear Trajectory Generation for Parallel Autonomy with a Dynamic Vehicle Model,” IEEE Transactions on Intelligent Transportation Systems, Oct. 2017 [Online]. Available: https://doi.org/10.1109/TITS.2017.2771351

  • W. Schwarting, J. Alonso-Mora, L. Paull, S. Karaman, and D. Rus, “Parallel Autonomy in Automated Vehicles: Safe Motion Generation with Minimal Intervention,” in 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore, Singapore, 2017 [Online]. Available: https://doi.org/10.1109/ICRA.2017.7989224