Inner Vision: Camera Based Proprioception for Soft Robots

Camera view
Edward Adelson, Daniela Rus

We will develop a soft robot hand with proprioception to be used in the HSR university challenge. It’s important for a robot to know the configuration of its own parts, i.e., to have proprioception. For example, in grasping an object, the robot positions the hand, and fingers carefully. In soft robotics the rules are relaxed: a soft hand can run open loop, mechanically adjusting for errors. But without proprioception the robot doesn’t know what it is doing.

[June-1-2018 to current]


  1. A. Amini, J. I. Lipton, and D. Rus, “Uncertainty Aware Texture Classification and Mapping Using Soft Tactile Sensors,” in IROS 2020 (accepted).
  2. Y. She, S. Q. Liu, P. Yu, and E. Adelson, “Exoskeleton-Covered Soft Finger with Vision-Based Proprioception and Tactile Sensing,” in 2020 IEEE International Conference on Robotics and Automation (ICRA), 2020, doi: 10.1109/ICRA40945.2020.9197369 [Online]. Available:
  3. L. Chin, M. C. Yuen, J. Lipton, L. H. Trueba, R. Kramer-Bottiglio, and D. Rus, “A Simple Electric Soft Robotic Gripper with High-Deformation Haptic Feedback,” in 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada, 2019, pp. 2765–2771, doi: 10.1109/ICRA.2019.8794098 [Online]. Available: