Navion: Enabling Autonomous Navigation of Miniaturized Robots



For updates on the Navion Project


Recent News


Abstract

Autonomous navigation of miniaturized robots (e.g., nano/pico aerial vehicles) is currently a grand challenge for robotics research, due to the need for processing a large amount of sensor data (e.g., camera frames) with limited on-board computational resources. In this project, we focus on the design of a visual-inertial odometry (VIO) system in which the robot estimates its ego-motion (and a landmark-based map) from on-board camera and IMU data.

In our RSS2017 paper, we argue that scaling down VIO to miniaturized platforms (without sacrificing performance) requires a paradigm shift in the design of perception algorithms, and we advocate a co-design approach in which algorithmic and hardware design choices are tightly coupled. Our contributions are four-fold. First, we discuss the VIO co-design problem, in which one tries to attain a desired resource-performance trade-off, by making suitable design choices (in terms of hardware, algorithms, implementation, and parameters). Second, we characterize the design space, by discussing how a relevant set of design choices affects the resource-performance trade-off in VIO. Third, we provide a systematic experiment-driven way to explore the design space, towards a design that meets the desired trade-off. Fourth, we demonstrate the result of the co-design process by providing a VIO implementation on specialized hardware and showing that such implementation has the same accuracy and speed of a desktop implementation, while requiring a fraction of the power.

VIO System Overview

VIO System Overview

Algorithm and Hardware Co-design

Algorithm and Hardware Co-design

In our VLSI2018 paper, we present an energy-efficient accelerator for VIO. The estimate of the drone’s trajectory and a 3D map of the environment is obtained by running a state-of-the-art algorithm based on non-linear factor graph optimization, which requires large irregularly structured memories and heterogeneous computation flow. To reduce the energy consumption and footprint, the entire VIO system is fully integrated on chip to eliminate costly off-chip processing and storage. This work uses compression and exploits both structured and unstructured sparsity to reduce on-chip memory size by 4.1x. Parallelism is used under tight area constraints to increase throughput by 43%. The chip is fabricated in 65nm CMOS, and can process 752x480 stereo images from EuRoC dataset in real-time at 20 frames per second (fps) consuming only an average power of 2mW. At its peak performance, Navion can process stereo images at up to 171 fps and inertial measurements at up to 52 kHz, while consuming an average of 24mW. The chip is configurable to maximize accuracy, throughput and energy-efficiency trade-offs and to adapt to different environments. To the best of our knowledge, this is the first fully integrated VIO system in an ASIC.

Navion Test Chip

Navion Test Chip

Memory Reduction Optimizations

Memory Reduction Optimizations



Videos

Navion Demos

Our Talk on Navion at Hot Chips-30


Press Coverage


Related Papers

  • P. Z. X. Li*, Z. Zhang*, S. Karaman, V. Sze, "High-throughput Computation of Shannon Mutual Information on Chip," Robotics: Science and Systems (RSS), June 2019. [ paper PDF | supplementary material PDF ]
  • Z. Zhang, T. Henderson, V. Sze, S. Karaman, "FSMI: Fast computation of Shannon Mutual Information for information-theoretic mapping," IEEE International Conference on Robotics and Automation (ICRA), May 2019. [ paper PDF | summary video ]
  • D. Wofk*, F. Ma*, T.-J. Yang, S. Karaman, V. Sze, "FastDepth: Fast Monocular Depth Estimation on Embedded Systems," IEEE International Conference on Robotics and Automation (ICRA), May 2019. [ paper PDF | project website LINK | summary video | code github ]
  • A. Suleiman, Z. Zhang, L. Carlone, S. Karaman, V. Sze, "Navion: A 2mW Fully Integrated Real-Time Visual-Inertial Odometry Accelerator for Autonomous Navigation of Nano Drones," to appear in IEEE Journal of Solid State Circuits (JSSC), VLSI Symposia Special Issue, Vol. 54, No. 4, April 2019. [ PDF ]
  • A. Suleiman, Z. Zhang, L. Carlone, S. Karaman, V. Sze, "Navion: An Energy-Efficient Visual-Inertial Odometry Accelerator for Micro Robotics and Beyond," IEEE Hot Chips: A Symposium for High-Performance Chips, August 2018. [ slides PDF | talk video ] Highlighted in EETimes
  • A. Suleiman, Z. Zhang, L. Carlone, S. Karaman, V. Sze, “Navion: A Fully Integrated Energy-Efficient Visual-Inertial Odometry Accelerator for Autonomous Navigation of Nano Drones,” IEEE Symposium on VLSI Circuits (VLSI-Circuits), June 2018. [ paper PDF | slides PDF | poster PDF ] Featured as a technical highlight at the conference.
  • Z. Zhang*, A. Suleiman*, L. Carlone, V. Sze, S. Karaman, “Visual-Inertial Odometry on Chip: An Algorithm-and-Hardware Co-design Approach,” Robotics: Science and Systems (RSS), July 2017. [ paper PDF | poster PDF | slides PDF | supplementary material PDF ] Highlighted in MIT News
* Indicates authors contributed equally to the work


Acknowledgement

This work was partially funded by the AFOSR YIP FA9550-16-1-0228 and by the NSF CAREER 1350685.