An autonomous car makes 30 driving decisions per second. Behind that is a pipeline of neural networks that turns photons and laser points into a safe trajectory. Let's open the black box.
Modern stacks use BEV (Bird's Eye View) transformers — cameras feed into a transformer that predicts a top-down occupancy grid. Tesla's HydraNets and Waymo's VectorNet do this jointly for lanes, traffic lights, and dynamic objects. LiDAR adds depth truth; radar adds velocity.
Humans are unpredictable. The AI predicts 6-8 possible trajectories for each agent (pedestrian, cyclist, car) and assigns probabilities. Waymo's MotionLM uses language-model-style transformers to forecast motion — trained on 40 million miles of real interaction.
For every real mile, companies simulate 10,000. NVIDIA Drive Sim, Waymo CarCraft and Tesla's simulation create edge cases — a child chasing a ball, a truck shedding cargo — that could never be safely tested live.
SYLVIE PIPKIN