Teleoperation: A Path Forward for Vehicle Autonomy Leave a comment

//php echo do_shortcode(‘[responsivevoice_button voice=”US English Male” buttontext=”Listen to Post”]’) ?>

Autonomous vehicles have progressed far beyond labs or test tracks and are now deployed worldwide on public and private roads. Thousands of delivery robots, driverless shuttles, long-haul trucks, middle-mile trucks, autonomous forklifts and tractors, and robotaxis operate daily in numerous pilots and even in commercial deployments worldwide. Most, if not all, of these vehicles rely on teleoperation to ensure they can complete their mission and not wait for a field crew to rescue them if they encounter an “edge case” (meaning something that developers did not anticipate fully) that they do not have the confidence to handle.

Yet some industry pundits would have you believe that teleoperation is not safe enough to be used on public roads.

That is because when they hear the term “teleoperation,” they think of a remote driver seated in a driving station, holding a steering wheel and pressing brake and acceleration pedals—with a car responding directly to their actions.

But is that really the case?

Let’s first define teleoperation and make an important clarification.

Broadly speaking, teleoperation means operating something from afar. Where it becomes tricky is breaking down what “operating” means and how it is technically done.

As noted, most people imagine a physical link between the remote terminal controls and the vehicle controls. This is far from reality, even with 5G.

No matter what type of teleoperation is implemented, the remote steering wheel is not instantaneously linked to the wheel, and the brake pedal does not directly apply the brakes. There are computing elements at both ends that process, transmit and execute actions.

The existence of these computing units, which may also have varying levels of autonomous capabilities, is what allows the creation of different levels of control by the remote operator. There is a whole scale of teleoperation modes, in which responsibility is allocated between the remote operator and the vehicle compute systems at different levels.

Along this scale, we distinguish between two high-level categories of remote operation.

Remote driving vs. tele-assist

In remote driving, the remote operator performs the dynamic driving task (DDT), which includes handling perception, localization, defining target steering and target speed. Even during remote operation, the vehicle itself performs some actions.

Under tele-assist, the autonomous driver is the entity performing the DDT and handles the majority of situations within the operational design domain. However, there are instances when the autonomous driver encounters unfamiliar or restricted scenarios, prompting it to seek assistance from a remote operator. The remote operator provides guidance, provides additional information or even implements a policy override, enabling the autonomous driver to successfully accomplish its mission.

Within these two high-level categories, there are several different teleoperation modes. The difference is the progression of the division of tasks between the remote operator and the autonomous driver.

There are three modes of teleoperation for driverless vehicles:

• Direct drive: The remote operator directly activates the actuators in the vehicle. The vehicle will stop if communication is lost. Typical use cases are geofenced environments and low-speed vehicles, such as robots, mining and agriculture.
• Low-level control: The remote operator controls trajectory and acceleration. The vehicle validates instructions and controls actuators. It is also in charge of safety by performing a minimal risk maneuver (MRM), i.e., a predefined action to ensure the safety of vehicle occupants and the surrounding environment. Based on the situation, an MRM can be a stop at the side of the road or proceeding slowly to a predefined area. Typical use cases are sidewalk delivery robots or remotely operated forklifts in yard operations.
• High-level control: The remote operator controls lanes and target speed. The vehicle performs low-level control of trajectory and acceleration and is in charge of safety and of executing an MRM if needed. A typical use case is driving on public roads in a relatively predictable scenario; for example, autonomous trucks on highways.

There are three tele-assist modes:

• Guide: The remote operator, if called on, assists the vehicle path planning process; for example, by providing waypoints or removing restrictions. The vehicle performs path planning. It is in charge of safety and executing an MRM if needed. A typical use case is driving in an urban environment.
• Advise: The remote operator responds to queries and assistance requests from the vehicle AI in near-real time. The vehicle performs the DDT and queries the remote operator in edge cases. It is in charge of safety and executing an MRM if needed. A typical use case is driving in complex situations, such as 24/7, all-season driving in a busy urban setting.
• Supervise: The remote operator monitors the vehicle and performs a safe stop if needed, usually due to regulatory requirements. The vehicle has full responsibility for driving, including safety and executing an MRM if needed.

The relevant mode of teleoperation depends on the use case, the design of the autonomous driver and other safety considerations.

It is worth noting that different teleoperation modes can be used during a specific drive, as required for the specific case and circumstances. For example, an autonomous vehicle may require remote driving with low-level control when exiting a warehouse yard, then high-level control when encountering unclear road markings on a highway or monitoring when driving in the vicinity of a school.

The path forward for autonomous vehicles: teleoperation

Now that we have clarified the difference between teleoperation, tele-assistance and remote driving, it is apparent that using the appropriate mode of teleoperation improves the safety of driverless vehicle deployments by providing a backup mechanism that can ensure mission accomplishment without disturbing public safety.