Dream · Open source · Gurgaon, India
Drive Gurgaon's real roads, in traffic that behaves like India's.
Two-wheelers squeeze past, autos cut in, people cross anywhere and cows stand in the lane. How often each of these happens comes from our dashcam footage of Delhi NCR. Dream is where driving software can learn Indian roads, in a dream of the world, before it meets the real one.
Why we built Dream
What makes Indian roads hard can be measured. Dream is our first test of that.
Most driving software learns on orderly roads. Our thesis is that the hard part of Indian roads is how often things happen, and that dashcam footage can measure it. So we measured it from our own footage, built a simulator on those numbers, and checked both against real roads and against a standard driving planner.
Dream will also hold what comes next: environments where driving software sees camera images from our footage, and world models trained on Indian roads.
Read our methodology: the thesis, what we tested, what is measured and what is assumed →
What it is built from
Real roads, measured traffic.
- The real road map
- Every road, junction and traffic signal in a stretch of Gurgaon about 5 km by 2 km, from OpenStreetMap.
- Traffic from our footage
- How often each kind of event happens comes from dashcam clips our team labelled by hand. Where we have no measurement yet, the code says so.
- Ready for AV teams
- The same simulator is a standard Python environment for training and testing driving software, with scores AV teams already use.
Does it match real roads?
12 of the 12 kinds of event we have measured happen about as often in the simulator as on real roads.
Each band is how often it happens on real Gurgaon roads. Our sample of footage is small, so it is a range, not a single number. The dot is the simulator: inside its band is a match.
27 of 100
Why it matters
routes ended in a crash for a textbook self-driving planner in Gurgaon traffic. In orderly traffic, 14.
We drove the same standard planner over the same routes twice: once in orderly, lane-keeping traffic and once in Gurgaon traffic. Software tested only in orderly traffic is not ready for Indian roads, and this is a free way to find out before a car does.
For AV teams
Use Dream in Python.
A Gymnasium environment that works with the reinforcement learning tools you already use. It follows the conventions of highway-env and MetaDrive, and scores planners the way nuPlan and CARLA do.
# install
pip install "thirdeyelabs[sim]"
# drive
import gymnasium as gym
import thirdeye_sim.env
env = gym.make("ThirdEye/Gurgaon-v0")
obs, info = env.reset(seed=0)
obs, reward, terminated, truncated, info = env.step(env.action_space.sample())
# score a planner, and check the traffic against our footage
python -m thirdeye_sim.evaluate
python -m thirdeye_sim.realism