NATIX is open-sourcing a third dataset on Hugging Face, adding a new layer to our real-world driving data: vehicle telemetry.
Our first public datasets combined multi-camera driving footage with GPS/GNSS, calibration, trip-level metadata, and long-tail scenarios curated from the visual data. This new release goes further by incorporating richer vehicle telemetry, including automotive-grade GPS, wheel angle, pedal position, brake state, turn-signal status, Autopilot state, and other key vehicle motion signals.
More importantly, we are now using that telemetry to identify long-tail driving scenarios directly from how the vehicle behaves.
The Safety-Critical Driving Events Dataset focuses specifically on events such as harsh braking, harsh acceleration, harsh turns, and aggressive driving patterns. The first release contains 32 minutes of real-world driving and 45 telemetry-detected safety-critical events across ten U.S. states, each time-stamped to the second and paired with the corresponding driving footage and vehicle context.
This adds a second approach to scenario mining within NATIX. Our previously released Edge Case Driving Dataset uses Vision-Language Models to analyze footage and identify unusual visual scenarios such as construction zones, adverse weather, road obstructions, and other long-tail events. The Safety-Critical Driving Events dataset uses vehicle telemetry to find significant moments based on how the vehicle itself moved.
The two approaches capture different signals, and together they provide a much richer way to find the moments that matter most for Physical AI.
Vehicle telemetry adds an important layer to autonomous driving data because it captures not only what the vehicle saw, but what the vehicle actually did.
When driving footage is synchronized with precise signals such as wheel angle, pedal position, vehicle speed, and automotive-grade GPS, researchers can condition World Models and end-to-end driving models on the vehicle's actual actions and trajectory. Instead of learning only how a driving scene evolves visually, a model can also learn how that scene changes in response to actions such as steering, braking, or accelerating.
Telemetry also gives NATIX another powerful way to find useful edge cases and long-tail scenarios inside thousands of hours of ordinary driving.
One way NATIX approaches this problem is through Vision-Language Models. VLMs understand the visual context surrounding an event, analyze driving footage, and identify what is happening visually: a pedestrian entering the road, an unusual construction setup, a vehicle blocking a lane, poor visibility, or another scenario worth extracting from the larger dataset.
The second approach uses vehicle telemetry. Like VLM processing, this is also part of the data-processing pipeline. The difference is the signal being analyzed.
Instead of asking a model to interpret every frame of video, telemetry processing can search signals such as acceleration, braking, steering behavior, pedal position, and vehicle state to identify moments where something significant happened. A sudden deceleration can flag a harsh-braking event. Steering and motion data can identify an unusually sharp maneuver. Combinations of signals can surface more complex aggressive-driving patterns.
Consider a harsh-braking event. Telemetry can tell us precisely when the vehicle decelerated sharply and how the vehicle behaved during the maneuver. But that signal alone cannot necessarily explain why the driver braked. That’s where a VLM can analyze the associated footage and describe what led to the situation.
The two methods are therefore complementary rather than competing. Telemetry can efficiently narrow thousands of hours of driving footage down to the moments worth investigating, without requiring a large multimodal model to inspect every second. Visual AI can then add the context needed to understand those moments in detail.
Vehicle telemetry finds the event. Visual AI explains the event.
That is particularly valuable for safety-critical moments. The most useful training data is often not another hour of ordinary highway driving, but the few seconds when something changed unexpectedly and the driver had to react.
By combining telemetry-based event detection with VLM-based scene understanding, NATIX can search its real-world driving network for very specific scenarios at scale while also producing richer training data for World Models and end-to-end driving models.
NATIX is not alone in treating access to Physical AI data as infrastructure.
Earlier this year, NVIDIA released its Open Physical AI Dataset, which includes 15 terabytes of robotics trajectories, along with plans to release tens of thousands of hours of multi-camera driving footage.
Around the same time, NVIDIA introduced Alpamayo, a family of open-source models for reasoning-based autonomous driving. Neither announcement was framed as charity. Both were framed as infrastructure.
The direction is becoming increasingly clear. The data bottleneck in Physical AI is not specific to NATIX, NVIDIA, or any individual autonomous-driving company. Systems that need to understand and interact with the physical world require enormous amounts of diverse real-world experience.
Opening parts of that data infrastructure allows researchers to train models, test new approaches, reproduce results, and build systems that would otherwise require their own large-scale collection fleets.
NATIX data is already being used by researchers and autonomous-driving organizations to build both World Models and end-to-end driving models.
At the University of Freiburg, researchers used more than 2,000 hours of NATIX driving footage, representing more than 40 percent of the training mix behind Orbis 2. The resulting hierarchical World Model demonstrated strong generalization to unseen driving data and outperformed NVIDIA Cosmos-v2.5 across several evaluated driving-world-model benchmarks.
NATIX is also working with Valeo to build one of the largest open-source multi-camera World Foundation Models. The project combines Valeo's world-model research with large-scale, real-world multi-camera driving data from NATIX, with an initial training foundation of over 5,000 hours of driving footage.
And through our work with the Autoware Foundation, NATIX is providing multi-camera driving data for Autoware's open-source end-to-end autonomous-driving model and the World Model components supporting it. The data will contribute to training, simulation, and validation across the open-source autonomy stack.
These projects cover different parts of the Physical AI stack, but they share the same dependency: diverse real-world driving data.
So far, these projects have primarily been built around video and its associated metadata. Vehicle telemetry introduces another layer of information and another way to extract the high-value moments hidden inside much larger collections of ordinary driving.
The value of a driving dataset is not simply how many hours of video it contains. It is how effectively those hours can be transformed into the moments that matter.
Our first dataset releases demonstrated how large-scale real-world footage and VLM-based scenario mining can expose the visual long tail of driving. The Safety-Critical Driving Events Dataset adds another dimension by using the vehicle's own telemetry to surface important events based on how the vehicle actually behaved.
Neither approach replaces the other.
Telemetry can efficiently identify when something significant happened. Vision-Language Models can help explain what happened and why. Used together, they provide a much richer representation of real-world driving than either signal alone.
With NATIX data already supporting World Model development, as well as open-source end-to-end driving, this third dataset expands the kinds of real-world signals researchers can access and build on.
The dataset is available now on Hugging Face. Researchers, academic institutions, and open-source teams looking for access to more of it can reach NATIX directly at dataset@natix.io.
