Sensors for robot perception

Sensors: Sensor is a device that measures or detects a property of the environment, or changes to a property.

Categorization of sensors: Exteroceptive (extero or surroundings), Proprioceptive (proprio or internal).

Mobile Robot Sensors. Source: Turns Out the Hardware in Self-Driving Cars Is Pretty Cheap

Type of Sensor Feature Weakness Future Trend More words
RGB Cameras Essential for robot to perceive environment with its rich semantics. ? HD, wide dynamic ranges Comparison Metrics: Resolution, Field of view, Dynamics range
Stereo Cameras This simulates human binocular vision and able to give scales/metrics for images. ? ? Definition: A type of camera with two or more lenses with a separate image sensor or film frame for each lens.
LiDAR It provides robots with detailed 3D scene geometry in the form of 3D point cloud. ? HD, low-cost, solid state LiDAR Definition: A sensing device that detects objects and maps their distances by illuminating a target with an optical pulse.
mmWave Radar Robust sensing tech. against poor visibility caused by fog, rain, dust and sand storms. ? single-chip, 4D imaging radar Definition: A sensing technology for detection of objects and providing the range, velocity and angle of these objects. It uses radio frequency technology operating in the spectrum between 30GHz and 300GH. More information: Doppler effect.
Ultrasonic / Sonar use robust mechanical wave, all-weather and all-lighting distance measurement Short-range (<5m) evolve to 3D ultrasonic, produce point clouds for close objects. Definition: An instrument that measures the distance to objects using ultrasonic sound waves.
GNSS ? ? ? Full name: Global Navigation Satellite Systems. Measuring Position and Velocity and Heading.
IMU ? ? ? Full name: Inertial measurement units. Measuring Angular rotation rate and Acceleration and Heading.
Wheel Odometry ? Speed accuracy and Position drift ? Tracks wheel velocity and orientation with a rotary encoder.

Synchronized Hardware

To synchronize different modules / sensors on a robot to provide a common clock. For autonomous driving, GPS and NTP (Network Time Protocol) time is often used as sources for a reference clock.

Time sync is crucial to multi-robot systems (e.g., a fleet of cars /drones) in which data will be shared and integrated for joint decision making.

Time sync is never a challenge, but the sync itself is a real challenge.

Computing hardware

Making decision

Takes in all sensory data and computes actions for the robot. Service robots have low requirements for the computing hardware (RPi 3/4 can be fine) but autonomous vehicles require powerful and customized computing units.

s21371309262022

Parallel computing due to heavy loads

Many tasks in autonomous robots can easily drain the compute resource, such as using images or LiDAR point cloud for image processing, object detection and mapping.

s21362709262022

Graphic Processing Unit (GPU)
Pending update

Field Programmable Gate Array (FPGA)
Pending update

Application Specific Integrated Chip (ASIC)
Pending update

Software decomposition for mobile robots

s21394509262022

Environment Perception
Pending update
s21453909262022

Environment Mapping
Pending update
s22120609262022

Motion Planning
Pending update
s22165609262022

Controller
Pending update
s22175709262022

System Supervisor
Pending update
s22181509262022

Question looking for answer

What are the basic components of a mobile robot?

How a mobile robot observes the physical word when moving around in the real world?

What is the necessary hardware and desirable software features of a mobile robot?

Design an omni-directional multi-sensor system for a mobile robot.

Describe the architecture of a typical (mobile) robotic system.

Understand the hardware and software components needed for a basic robot.

Why we cannot use lidar in harsh sunlight?

Smaller topics

Can we use ultrasonic sensor to measure the speed?
Maybe, Amazon has some research on this topic.

Is there any good purpose to do so?
In water.

Can we use other mechanic wave in autonomous vehicles? What about infrasonic?
Research project is possible.

What’s the purpose to have more lenses?
Generally on a robot, if a stereo camera have more than 2 cameras, only 2 of them are used for stereo vision. 

Take the Intel D435 for example. The 2 cameras for stereo are black&white cameras. A third camera is added for RGB vision.

There are analogue cameras with more than 2 cameras. They are used for photography and artistic creation, not for robotics.

What is separate film frame?

Film frame is where the film is located on a film camera.

A film frame on a film camera equals to a sensor (CCD or CMOS) on a digital camera.

This definition of stereo cameras also include stereo film cameras. But they are not used in robotics.

Supplement material


Origin: Dr. Chris Lu (Homepage)
Translate + Edit: YangSier (Homepage)

:four_leaf_clover:碎碎念:four_leaf_clover:
Hello米娜桑,这里是英国留学中的杨丝儿。我的博客的关键词集中在编程、算法、机器人、人工智能、数学等等,点个关注吧,持续高质量输出中。
:cherry_blossom:唠嗑QQ群兔叽的魔术工房 (942848525)
:star:B站账号白拾Official(活跃于知识区和动画区)