Scenario based testing is how autonomous driving teams turn rare situations into tests they can run again and again. Instead of driving millions of miles and hoping a dangerous moment appears, they describe that moment precisely. They then recreate it in simulation and vary it to probe how a system behaves. A single recorded cut-in on a highway can become hundreds of test cases, each with a slightly different speed, gap or road layout.
It is the step that gives the effort of finding edge cases lasting value. Finding a rare event once is useful. Being able to test against it every time the software changes is what actually makes a system safer.
A scenario, in this sense, describes how a driving situation unfolds over time. It covers where it happens, who is involved and what each road user does. Scenario based testing runs a driving system through many such descriptions, usually in simulation, and checks whether it behaves safely in each one.
The approach complements real-world driving rather than replacing it. Road testing shows how a system performs in the conditions it happens to meet. Scenario based testing lets engineers choose the conditions, including dangerous ones that would be unethical or impractical to stage on a public road.
Most valuable test scenarios start as real events. A team finds an interesting moment through scenario mining, for example a near miss flagged during a naturalistic driving study style analysis. They then extract the essential elements: the road layout, the positions and speeds of each vehicle, and the sequence of actions.
Those elements are written in a format that simulation tools can read. The most widely used open standard is ASAM OpenSCENARIO. It defines a file format for the dynamic content of driving and traffic simulators, such as the maneuvers of vehicles and pedestrians. According to ASAM's standard page, the current XML version is 1.3.1, released in November 2024, and it is free to download. Road networks are described separately in a companion standard, ASAM OpenDRIVE.
The standard's documentation notes that a scenario can be based on trajectories derived from a recorded driving maneuver, which is exactly how real events become simulation tests.
Teams build scenario libraries from two sources. Designed scenarios are written by engineers to cover known requirements, such as the test cases defined by regulators and consumer testing programs. They are systematic and easy to explain, but they only contain what someone thought to include.
Recorded scenarios come from real driving. They capture the odd combinations and unexpected behaviors that nobody would think to design. That is why they are so valuable for the long tail. Their weakness is coverage: you only get the events that happened to be recorded and found.
The trade-off mirrors the wider debate we covered in synthetic data vs real data. The strongest libraries combine both. Designed scenarios provide structure and regulatory coverage, while recorded ones keep the library honest about how the world actually behaves. Both feed into scenario generation, where new variations are created from a base description.
The real power of scenario based testing comes from variation. Once a situation is described in a structured way, its parameters can be changed systematically. Typical parameters include:
Running hundreds of variations shows where a system's behavior starts to break down. A planner might handle a cut-in at a three-second gap but fail at two seconds in the rain. Finding that boundary in simulation is far cheaper and safer than finding it on the road.
There is no single number. The right size of a scenario library depends on the operating area, the type of system and the risks that matter most. A highway driving feature needs deep coverage of merges, cut-ins and lane closures. An urban robotaxi needs far more variety in pedestrians, cyclists and junction types.
A more useful question is whether the library covers the situations that actually occur where the system will drive. Teams track this by listing scenario types and counting how many verified examples and variations exist for each. Gaps in that list show where more mining or more recording is needed. Over time, every failure found in testing or on the road should become a new scenario. The library then grows in the places where the system has shown weakness.
Traditional simulators rely on hand-built 3D environments and scripted behavior. Generative world models are starting to change that by producing realistic sensor video directly from a scenario description. We explored that shift in our article on generative simulation replacing traditional AV simulators. The result is test scenes that look much closer to real camera footage.
Those models still need real data to learn from, and real scenarios to stay grounded. NATIX's partnership with MaprGo shows one way to connect the two. As described in our MaprGo announcement, the collaboration turns real driving footage into simulation-ready scenarios and maps in open formats such as OpenSCENARIO. Teams can start from a real driving dataset and end with tests that reflect how roads and road users actually behave.
Scenario based testing is the bridge between finding the long tail and proving a system can handle it. The better the real scenarios feeding it, the more that proof is worth.
