Shadow mode is a way of testing a driving model on real roads without letting it drive. The model runs silently in the background of a production vehicle while a human is in control. It watches the same scene, decides what it would do, and the system logs the moments when its decision differs from the driver's. Each of those disagreements is a candidate edge case, found in everyday traffic rather than on a test track.
For fleets with many vehicles on the road, this turns ordinary driving into a continuous search. The target is every situation a model does not yet handle well.
The idea is simple. A new model, or a new feature, is deployed to vehicles but not connected to the steering, brakes or accelerator. It receives the same camera and sensor inputs as the active system. It produces outputs, such as a planned path or a detection, but those outputs are only recorded, never acted on.
Because the human driver is still in control, the comparison has a natural reference point. Suppose the model would have braked and the driver did not, or the driver swerved around something the model ignored. Either moment is worth a closer look. At its 2019 Autonomy Day, Tesla said it runs its networks in shadow mode before they have any control, according to coverage of the event.
In practice, testing this way usually combines the silent model with a set of triggers that decide which moments are worth uploading. Recording everything would overwhelm storage and review teams, so the fleet only sends back clips that match specific conditions.
Tesla gave the clearest public description of this setup. At the CVPR 2021 Workshop on Autonomous Driving, its then head of AI said the fleet ran 221 manually written triggers. Each one captured a situation of interest, as summarized in this account of the talk. Some triggers fired on disagreement between the model and the driver. Others fired on specific scene types the team wanted more examples of.
A second use is checking a new model against a trusted reference. One reported example compared distances estimated by a camera-based system with readings from ultrasonic sensors. The sensors served as the reference during the transition, according to this explainer on Tesla's approach. Disagreements in that setting point to errors in the new system rather than to unusual road situations.
In summary, silent testing is strong and weak in predictable ways:
It is very good at finding situations where a model's decision differs from a careful human's. Those disagreements are also a natural complement to disengagements, the moments when a safety driver takes over from an active system.
It also has blind spots. It can only surface situations that the fleet encounters. A fleet concentrated in a few regions will mostly find edge cases from those regions, which limits how representative its fleet data can be. It depends on the human driver being a good reference, which is not always true. We explored why imitating humans has limits in the expert driver problem. And it needs carefully designed triggers, which means someone must anticipate what to look for. Scenes where the model and driver happen to agree, even if both are wrong, stay invisible.
The approach also differs from older research methods. The naturalistic driving study approach looks for physical signs that something went wrong, such as hard braking. Shadow mode looks for disagreement between a model and a person. The two methods find overlapping but different sets of events. That is why mature teams use both, as we describe in our pillar on scenario mining.
The technique becomes even more useful as driving systems move toward end-to-end autonomous driving, where one model maps camera input directly to driving behavior. In those systems, comparing the model's planned trajectory with the human's actual path is a natural way to find weaknesses. Reasoning models add another layer, because they can explain why they would have acted differently, which helps reviewers sort real problems from harmless differences. We look at that shift in our article on reasoning in autonomous driving AI.
The whole method assumes you own a fleet. Most teams do not, or their fleet covers only a few cities. That is the main limit on how many edge cases they can find this way.
Large collections of real driving from other sources offer a way around it. Teams can run a model offline against recorded multi-camera footage and compare its outputs with what the human drivers did. The disagreements can then be mined much as they would be on the road. The open-source Autoware project took this direction when it began using multi-camera data for its end-to-end driving stack, with NATIX supplying real-world footage from everyday drivers. Similar autonomous driving data lets teams test against regions and conditions their own vehicles have never driven.
Shadow mode showed the industry that every mile driven by a human can double as a test. The next step is making that test possible for teams without a million cars of their own.
