The SAE levels of autonomous driving are the industry's shared way of describing how much a car drives itself. Most explanations stop at who holds the steering wheel. That is useful, but it leaves out the question engineering teams care about most: what does each level demand from the data used to build it?
The short answer is that data needs grow with responsibility. As the system takes over more of the driving, and more of the failures, its training data must cover more of the world in more detail. This guide walks through the levels with that lens.
The levels come from SAE J3016, the standard that defines driving automation. It describes six levels, from 0 to 5.
Levels 0 to 2 are usually grouped as driver assistance, or ADAS, while Levels 3 to 5 count as automated driving. Across the SAE levels of autonomous driving, the key dividing line is who acts as the fallback when something goes wrong.
Level 2 autonomous driving is the most common form of automation on the road today, and adaptive cruise control combined with lane centering is a typical example. The driver must keep watching the road at all times.
"Level 2+" is a market label, not an official SAE level. It describes Level 2 systems with richer features. Examples include hands-free highway driving, automatic lane changes and navigation-guided driving. The driver is still responsible, which is why US regulators still treat these systems as Level 2 ADAS in NHTSA's crash reporting order.
From a data point of view, Level 2 systems need broad, reliable coverage of common situations, with precise labels for lanes, vehicles and signs across many markets. They must also avoid false alarms, because surprising the driver erodes trust quickly.
Level 3 marks the first real handover of responsibility. In defined conditions, such as slow highway traffic, the system drives and the driver may look away. When the system reaches the edge of its conditions, it asks the driver to take back control.
This handover shapes the data. A Level 3 system must recognize reliably when it is leaving its comfort zone. It also needs to give the driver enough warning to respond. Training data must therefore include many examples of the boundary itself, such as traffic speeding up, weather changing or lane markings fading.
Level 4 autonomy removes the driver from the fallback role within a defined operating area. Robotaxi services are the best-known example. Inside that area, the system must handle every situation on its own, including its own failures.
That is a very different data challenge. The industry describes the operating area as the operational design domain. Inside it, coverage must be deep. Every type of junction, weather condition and road user should appear many times. Rare events matter most, because there is no human to catch the mistake.
Level 5 would extend this to every road and condition. No system has reached it yet, and for practical purposes, today's frontier is Level 4.
Looking across the SAE levels of autonomous driving, three data demands rise together.
The first is coverage. Lower levels need breadth across markets, while higher levels need depth inside a domain, with a full view of the scene around the vehicle.
The second is labeling depth. Lower levels can rely on 2D objects, lanes and signs, while higher levels need 3D structure, tracking over time and an understanding of what other road users will do next.
The third is edge-case density. Lower levels mostly need protection against false activations, while higher levels need exposure to situations the system has rarely or never seen. We compare these differences in depth in our guide to ADAS data and full autonomy.
The SAE levels of autonomous driving are therefore a useful starting point for any data plan. The most common mistake is to source data for the wrong level. A Level 2 team that buys deep coverage of one city may find it lacks the breadth its feature needs. A Level 4 team that buys broad highway footage may find it lacks the rare urban events that decide safety.
A practical approach is to start from the fallback. If the driver is the fallback, prioritize breadth, precision and low false-alarm rates. If the system is the fallback, prioritize depth in the domain, multi-camera views of the whole scene and rare events. In both cases, the data should come from cameras close to the ones the system will use in production.
This is where large volumes of real-world driving data help. NATIX collects multi-camera footage from factory Tesla cameras across the USA, Europe and Japan. It records driver actions such as steering and braking alongside the video. Teams can cut that data by region, road type or scenario to match the level they are building.
The SAE levels of autonomous driving describe who is responsible. The data behind each level has to reflect that responsibility. The higher the level, the more of the real world the system must have seen before it is trusted to drive.
