Sensors for robot perception

Sensors: Sensor is a device that measures or detects a property of the environment, or changes to a property.

Categorization of sensors: Exteroceptive (extero or surroundings), Proprioceptive (proprio or internal).

Mobile Robot Sensors. Source: Turns Out the Hardware in Self-Driving Cars Is Pretty Cheap

Type of Sensor Feature Weakness Future Trend More words
RGB Cameras Essential for robot to perceive environment with its rich semantics. ? HD, wide dynamic ranges Comparison Metrics: Resolution, Field of view, Dynamics range
Stereo Cameras This simulates human binocular vision and able to give scales/metrics for images. ? ? Definition: A type of camera with two or more lenses with a separate image sensor or film frame for each lens.
LiDAR It provides robots with detailed 3D scene geometry in the form of 3D point cloud. ? HD, low-cost, solid state LiDAR Definition: A sensing device that detects objects and maps their distances by illuminating a target with an optical pulse.
mmWave Radar Robust sensing tech. against poor visibility caused by fog, rain, dust and sand storms. ? single-chip, 4D imaging radar Definition: A sensing technology for detection of objects and providing the range, velocity and angle of these objects. It uses radio frequency technology operating in the spectrum between 30GHz and 300GH. More information: Doppler effect.
Ultrasonic / Sonar use robust mechanical wave, all-weather and all-lighting distance measurement Short-range (<5m) evolve to 3D ultrasonic, produce point clouds for close objects. Definition: An instrument that measures the distance to objects using ultrasonic sound waves.
GNSS ? ? ? Full name: Global Navigation Satellite Systems. Measuring Position and Velocity and Heading.
IMU ? ? ? Full name: Inertial measurement units. Measuring Angular rotation rate and Acceleration and Heading.
Wheel Odometry ? Speed accuracy and Position drift ? Tracks wheel velocity and orientation with a rotary encoder.

Synchronized Hardware

To synchronize different modules / sensors on a robot to provide a common clock. For autonomous driving, GPS and NTP (Network Time Protocol) time is often used as sources for a reference clock.

Time sync is crucial to multi-robot systems (e.g., a fleet of cars /drones) in which data will be shared and integrated for joint decision making.

Time sync is never a challenge, but the sync itself is a real challenge.

Computing hardware

Making decision

Takes in all sensory data and computes actions for the robot. Service robots have low requirements for the computing hardware (RPi 3/4 can be fine) but autonomous vehicles require powerful and customized computing units.


Parallel computing due to heavy loads

Many tasks in autonomous robots can easily drain the compute resource, such as using images or LiDAR point cloud for image processing, object detection and mapping.


Graphic Processing Unit (GPU)
Pending update

Field Programmable Gate Array (FPGA)
Pending update

Application Specific Integrated Chip (ASIC)
Pending update

Software decomposition for mobile robots


Environment Perception
Pending update

Environment Mapping
Pending update

Motion Planning
Pending update

Pending update

System Supervisor
Pending update

Question looking for answer

What are the basic components of a mobile robot?

How a mobile robot observes the physical word when moving around in the real world?

What is the necessary hardware and desirable software features of a mobile robot?

Design an omni-directional multi-sensor system for a mobile robot.

Describe the architecture of a typical (mobile) robotic system.

Understand the hardware and software components needed for a basic robot.

Why we cannot use lidar in harsh sunlight?

Smaller topics

Can we use ultrasonic sensor to measure the speed?
Maybe, Amazon has some research on this topic.

Is there any good purpose to do so?
In water.

Can we use other mechanic wave in autonomous vehicles? What about infrasonic?
Research project is possible.

What’s the purpose to have more lenses?
Generally on a robot, if a stereo camera have more than 2 cameras, only 2 of them are used for stereo vision. 

Take the Intel D435 for example. The 2 cameras for stereo are black&white cameras. A third camera is added for RGB vision.

There are analogue cameras with more than 2 cameras. They are used for photography and artistic creation, not for robotics.

What is separate film frame?

Film frame is where the film is located on a film camera.

A film frame on a film camera equals to a sensor (CCD or CMOS) on a digital camera.

This definition of stereo cameras also include stereo film cameras. But they are not used in robotics.

Supplement material

Origin: Dr. Chris Lu (Homepage)
Translate + Edit: YangSier (Homepage)

:cherry_blossom:唠嗑QQ群兔叽的魔术工房 (942848525)