A look under our trunk: what’s in our computehttps://waymo.com/blog/2026/08/look-under-our-trunk/
1%
Who will be #2 robotaxi in the US? : r/SelfDrivingCars - Reddithttps://www.reddit.com/r/SelfDrivingCars/comments/1r9225s/who_will_be_2_robotaxi_in_the_us/
1%
Tesla Robotaxi vs Waymo: I Analyzed Real Fleet Data ...https://www.youtube.com/watch
1%
6 Best Self-Driving Cars in the US (2026) - Fifth Level Consultinghttps://fifthlevelconsulting.com/best-self-driving-cars-in-the-us/
1%
NVIDIA Hyperion Becomes the Global Platform for a Robotaxi-Ready World | NVIDIA Newsroomhttps://nvidianews.nvidia.com/news/nvidia-drive-hyperion-becomes-the-global-platform-for-a-robotaxi-ready-world
1%
Top 5 Self-Driving Car Companies in 2025 - Shapiro Law Teamhttps://shapirolawaz.com/2025/05/29/self-driving-car-companies/
1%
How our cars drive - Waymo Helphttps://support.google.com/waymo/answer/9190838
When building or evaluating a self-driving technology stack for a commercial robotaxi fleet , the industry generally agrees that "the best" stack is not just a collection of AI models, but a three-computer architecture coupled with a robust, safe sensor suite . Commercial leaders like Waymo, Motional , and Pony.ai utilize a mature, modular approach, while others lean toward pure vision-based end-to-end neural networks.
An optimal Level 4 (L4) robotaxi technology stack breaks down into key structural layers:
1. The Three-Computer Infrastructure
The backbone of a modern robotaxi fleet extends far beyond the hardware in the trunk:
Data Center Training: High-performance clusters (such as ) running massive foundational Vision-Language-Action (VLA) models and deep learning pipelines on petabytes of fleet-derived edge-case data.
Simulation & Validation: Virtual environments (like NVIDIA Omniverse ) to test software updates against billions of simulated miles, reproducing rare corner cases safely before pushing code to production.
In-Vehicle Compute: High-throughput, low-latency automotive-grade edge computers (such as NVIDIA DRIVE AGX ) capable of real-time sensor fusion and path planning redundancy.
2. Sensor Suite: Multi-Modal vs. Vision-Only
The Multi-Modal Approach (Waymo, Cruise, Baidu, Motional): Combines long-range LiDAR, radar, and high-resolution cameras . This redundancy is widely considered the safest route for dense urban L4 robotaxis because LiDAR provides independent, millimeter-accurate depth and geometry data regardless of blinding sunlight, heavy rain, or glare.
The Vision-Centric Approach (Tesla): Relies purely on surround-video cameras paired with neural networks. While cheaper and theoretically more scalable for consumer passenger vehicles, it requires exponentially more compute and data to handle edge-case depth perception without active sensors.
3. Software Architecture Layers
Perception: Real-time object detection, tracking, and classification (pedestrians, cyclists, erratic drivers) using deep learning combined with geometric map matching.
Prediction & Localization: Anticipating the future trajectories of dynamic objects around the vehicle while fixing precise vehicle coordinates via HD maps and GNSS/inertial sensor fusion.
Planning & Control: Behavior generation (deciding when to edge around a double-parked delivery truck) and smooth trajectory execution governing steering, throttle, and braking.
Fleet Teleoperations & Management: A cloud backend for route optimization, demand analysis, remote path-assistance (when a vehicle encounters a confusing construction zone), and health diagnostics.
If you're designing or analyzing a specific deployment, let me know:
Are you targeting dense urban centers with geofenced HD maps, or a broader operational design domain (ODD)?
Are you leaning toward a multi-modal (LiDAR + Camera) or vision-only hardware approach?
If you mean the best stack to build or buy for a commercial L4 robotaxi fleet in 2026, I’d favor a multimodal, redundant, vertically integrated stack built around lidar + radar + cameras, with high-end automotive compute and a strong simulation/data loop.
The important distinction is that “best” depends on whether you want to own the autonomy IP or launch a fleet quickly.
My recommended architecture
Layer
Recommended approach
Why
Sensors
360° lidar + radar + HDR cameras + ultrasonics
Redundancy across sensing modalities and weather conditions
Remote assistance, dispatch, diagnostics, cleaning/charging and recovery
Robotaxi economics depend on operations as much as autonomy
The strongest current reference architectures
1. Waymo-style stack — strongest proven template
Waymo's architecture is unusually comprehensive: lidar, radar and cameras feed an integrated onboard computer, with detailed maps and AI-based perception/planning. Waymo says its latest-generation system has accumulated more than 200 million fully autonomous miles, and as of September 2026 it was offering fully autonomous rides in 14 cities.
The big lesson isn't necessarily “copy Waymo.” It's co-design the vehicle, sensors, compute, autonomy software, mapping and fleet operations as one system.
2. NVIDIA Hyperion — strongest starting point for a new fleet
For a company that doesn't want to spend a decade developing every layer itself, NVIDIA's current Hyperion platform is probably the most interesting foundation. It combines automotive compute, cameras, radar, lidar, ultrasonics, DriveOS/DRIVE AV, safety infrastructure, simulation and training tooling. Hyperion 10 specifies dual Thor compute, 14 cameras, nine radars and lidar, with an ASIL-D-capable architecture.
There's also meaningful ecosystem validation: Uber announced plans for at least 5,000 Level-4 vehicles based on the NVIDIA architecture, while NVIDIA says multiple automakers and AV developers are building around Hyperion.
That makes it particularly attractive if your objective is fleet deployment rather than inventing an autonomous-driving computer from scratch.
Zoox takes the other major approach: design the vehicle around autonomy and ride-hailing rather than retrofit an existing car. Its production robotaxi uses cameras, lidar, radar and long-wave infrared, with a bidirectional vehicle architecture and detailed mapping/localization.
That's compelling for a very large dedicated fleet, although it requires substantially more vehicle engineering.
4. Aurora-style approach — interesting if you want a portable autonomy stack
Aurora combines lidar, radar and cameras with its “Verifiable AI” approach, virtual testing and a common autonomous-driving platform intended to move across different vehicle types.
That architecture is particularly interesting if your business might eventually operate both robotaxis and autonomous commercial vehicles.
What I would actually build
For a new robotaxi company, I'd choose something roughly like:
fleet data → automatic event mining → labeling → training → closed-loop simulation → safety evaluation → shadow deployment → OTA
That last loop is arguably as important as the neural network itself. Uber, for example, is building a dedicated data-collection operation intended to generate millions of miles of high-fidelity AV data per month for its autonomy partners.
One thing I would not optimize for
I wouldn't select a stack primarily because it has the largest neural network or the fewest sensors.
For a robotaxi, the objective is not “maximum AI benchmark performance.” It's something closer to:
That favors redundancy, deterministic safety mechanisms, excellent simulation, strong fleet operations and rapid iteration.
So, if you're asking me to choose a starting point for a hypothetical company in 2026:
Best buy/build foundation: NVIDIA Hyperion-class hardware + your own autonomy software/data/safety layer.
Best architectural model to study: Waymo.
Best purpose-built vehicle philosophy: Zoox.
I would not simply buy an off-the-shelf “self-driving system” and treat it as the whole robotaxi stack. The defensible asset is the combination of autonomy software, fleet data, simulation/validation, maps, operational tooling and safety case—not merely the vehicle computer.
If you're evaluating this from an engineering/business perspective, I can also break down a concrete $50M / $250M / $1B robotaxi tech stack, including sensors, compute BOM, cloud GPU requirements, engineering headcount, fleet size and estimated per-mile costs.
When designing or evaluating a self-driving technology stack for a commercial robotaxi fleet, the industry is broadly divided into two major architectural philosophies . There isn't a single "best" consensus, but rather a fierce debate between a sensor-fusion/HD-mapped approach and an end-to-end vision-only approach.
An optimal, production-grade robotaxi stack comprises several core layers, heavily influenced by the leading deployments today (such as Alphabet's Waymo One, Baidu Apollo Go , and Tesla's Cybercab ecosystem):
1. Perception Layer (The "Eyes & Ears")
The Redundant Sensor Suite (Waymo/Zoox/Baidu approach): Combines long- and short-range LiDAR (light detection and ranging), automotive-grade radar , and high-resolution cameras . LiDAR provides active, millimeter-precise depth perception independent of ambient lighting, acting as a crucial safety net in heavy glare, fog, or unmapped edge cases.
The Vision-Only Approach (Tesla approach): Relies strictly on high-definition surround cameras coupled with massive neural networks (end-to-end AI). Proponents argue that if humans drive using passive optical vision, machines can too, drastically lowering unit production costs. However, critics note it remains more susceptible to blinding sunlight, heavy precipitation, or sensor obstruction.
2. Localization & Mapping Layer
High-Definition (HD) Pre-Mapping: Used by traditional autonomous vehicle (AV) leaders. Vehicles cross-reference real-time sensor data against hyper-detailed 3D maps of a city down to the centimeter (curb locations, lane lines, traffic light coordinates). This reduces real-time compute requirements for basic environment geometry.
Real-Time SLAM (Simultaneous Localization and Mapping): Vision-first and generalized AI stacks rely less on rigid static HD maps and more on dynamic, real-time spatial understanding to handle road construction, detours, and unmapped territories dynamically.
3. Planning & Control (The "Brain")
Behavior Prediction & Path Planning: Modern stacks utilize Foundation Models / Embodied AI to predict the future trajectories of pedestrians, cyclists, and erratic human drivers seconds in advance.
Modular vs. End-to-End Neural Networks: Traditional stacks use a modular pipeline (Perception → Prediction → Planning → Control), making it easier to debug specific errors. The cutting-edge shift favors End-to-End (E2E) deep learning , where raw sensor data goes directly into a neural network that outputs steering, acceleration, and braking commands, yielding much smoother, human-like driving behavior.
4. Compute & Hardware Infrastructure
High-Performance Automotive Compute: Heavy reliance on centralized automotive supercomputers (such as NVIDIA DRIVE platforms or proprietary ASICs like Tesla's FSD computer) capable of processing hundreds of teraflops to petaflops of sensor data in milliseconds with automotive-grade thermal and functional safety (ASIL-D) compliance.
Hardware Redundancy: Dual-string power, braking, and steering systems are mandatory for driverless Level 4 (L4) operation so that if a primary computer or actuator fails, a secondary system safely brings the vehicle to a pullover or stop.
5. Fleet Operations & Teleoperations (The "Remote Safety Net")
Remote Assistance & Path Confirmation: A complete robotaxi stack requires a robust cloud backend. When a vehicle encounters an ambiguous edge case (e.g., a confusing police checkpoint or blocked construction lane), it doesn't "guess"—it stops and pings a human remote-assistance center for high-level path authorization or teleoperation guidance.
The Verdict: Which Philosophy Wins?
For Commercial Viability and Rapid Urban Launch Today: The sensor-heavy, HD-mapped stack (Waymo model) is proven safest and most legally compliant for driverless, un-supervised public ride-hailing in dense metropolitan areas.
For Mass-Market Scalability and Unit Economics: The vision-centric, generalized AI stack (Tesla model) aims for a lower hardware cost per vehicle, though it faces steeper regulatory hurdles and rigorous public scrutiny regarding edge-case safety without LiDAR fallback.
Are you looking to build a stack for a specific operating environment (like a tightly geo-fenced urban center vs. open highways), or are you more interested in the cost trade-offs between LiDAR and camera-only architectures?
redundant multimodal sensing + centralized high-performance compute + an independently safety-rated fallback + a large simulation/data loop
As of September 2026, there are two particularly credible reference points: Waymo’s vertically integrated Driver and NVIDIA’s Hyperion platform. Waymo has accumulated nearly 200 million fully autonomous miles and its sixth-generation system combines lidar, cameras, and imaging radar. WaymoWaymo NVIDIA, meanwhile, is explicitly packaging Hyperion as a production-oriented L4 reference architecture, with standardized sensors, redundant compute, and an end-to-end development/validation stack.
Separates strategic decisions from safety-critical control
Control
Independent real-time vehicle controller
Keeps actuation predictable even if higher-level AI fails
Safety
Independent safety monitor/fallback planner
Detects invalid AI outputs and brings vehicle to a minimal-risk state
Maps
HD map + continuously updated semantic/geometric layers
Particularly valuable for initial deployment and difficult intersections
Fleet cloud
Data ingestion → labeling → training → simulation → validation → OTA
The fleet becomes a continuous learning/validation system
Remote operations
Human assistance for exceptional situations, not remote driving by default
Handles ambiguous edge cases without putting latency-critical driving in the cloud
1. Sensor suite: don't go camera-only
For a robotaxi, I'd use all three major sensing modalities:
Cameras: semantics, traffic lights, signs, lane markings and object classification.
Lidar: highly accurate 3D geometry and object localization.
Radar: velocity and robustness in rain, fog, dust and other degraded visibility.
That is essentially the philosophy used by Waymo: its current Driver combines lidar, cameras and radar, with each providing complementary information.
NVIDIA's current Hyperion 10 reference architecture similarly specifies 14 cameras, 9 radars, 1 lidar, 12 ultrasonics and 4 interior cameras, backed by two Thor SoCs.
The exact sensor count isn't sacred. Redundancy and coverage are.
2. Compute: NVIDIA Hyperion is a compelling starting point
If I were starting a robotaxi program today and didn't already have Waymo-scale proprietary hardware/software, I'd seriously consider NVIDIA Hyperion rather than designing the entire compute platform myself.
NVIDIA's current Hyperion 10 architecture uses two DRIVE AGX Thor systems, with NVIDIA claiming up to 1,000 INT8 TOPS and 2,000 FP4 TFLOPS per SoC, alongside redundant compute and ASIL-D-oriented safety architecture.
That gives you an important architectural advantage:
You can spend your engineering effort on the parts that actually differentiate the fleet—perception, prediction, planning, data and safety validation—instead of reinventing the automotive compute platform.
There is also evidence that this ecosystem is becoming commercially significant: NVIDIA says automakers, AV software companies and mobility providers are building L4 fleets around Hyperion, and Reuters reported this week that Lucid/Bolt's planned European robotaxi fleet will use it.
The crucial design decision is not letting one giant neural network be the only thing between sensors and steering/braking.
I'd absolutely use large multimodal models where they provide value. But the system needs independent mechanisms capable of saying:
"That proposed trajectory is unsafe. Reject it."
That distinction matters enormously when you're operating thousands of vehicles without a driver.
4. HD maps + AI, rather than "maps OR AI"
I'd use HD maps as a prior, not as the vehicle's source of truth.
Waymo explicitly describes its HD maps this way: the map provides prior information that helps the system, particularly in complicated roads and poor visibility, while the vehicle's sensors remain responsible for what is actually happening now.
That gives you:
Map says:
"This intersection normally has three lanes and a left-turn pocket."
Sensors say:
"Construction has moved the lanes today."
Planner says:
"Trust current perception; deviate from the map."
That's much more robust than either pure map dependence or pretending maps have no value.
5. The real moat: the data engine
For a fleet operator, I would actually rank this above the neural-network architecture.
Build:
fleet → interesting events → automatic labeling → simulation → training → validation → OTA → fleet
You want the system continuously harvesting:
disengagement-like events
near collisions
unusual pedestrian behavior
construction
emergency vehicles
weird road geometry
adverse weather
sensor failures
planner uncertainty
novel objects
situations where humans behave unexpectedly
Then replay those situations in simulation thousands or millions of times before shipping a new model.
NVIDIA's current platform explicitly incorporates this concept through its AV Data Factory and simulation/training pipeline.
6. What I'd buy vs. build
Buy/license:
Automotive compute
Basic vehicle platform
Radar/lidar/camera hardware
Automotive OS/middleware where appropriate
Functional-safety infrastructure
Simulation infrastructure where it is commodity
Build internally:
Sensor fusion
Perception models
Prediction
Planning
Safety supervisor
Fleet data engine
Scenario mining
Maps and map-update pipeline
Validation methodology
Remote-assistance software
Operational design domain logic
That's essentially the compromise between Waymo's vertically integrated approach and NVIDIA's ecosystem approach.
My practical architecture choice
If I had to greenfield a commercial fleet in 2026, I'd start with:
Then I'd progressively replace commodity pieces with proprietary technology as fleet volume justifies it.
The biggest mistake would be optimizing for minimum sensor cost or maximum AI benchmark performance. A robotaxi is ultimately a safety-critical fleet operation, so I'd optimize for measurable reliability per vehicle-mile, graceful degradation, validation coverage, and operating cost.
And one important caveat: today's commercial systems still encounter significant edge cases. For example, Waymo temporarily suspended San Antonio service after flooding incidents and subsequently made software changes to its system. San Antonio Express-NewsAxios That illustrates why the fleet-level safety/operations layer is just as important as the driving model itself.
If you're actually designing a robotaxi company, I can also lay out a concrete $10M / $100M / $1B technology roadmap, including hardware BOM, compute requirements, engineering headcount, cloud costs, simulation infrastructure, and the software modules I'd build first.
That distinction matters enormously when you're operating thousands of vehicles without a driver.
4. HD maps + AI, rather than "maps OR AI"
I'd use HD maps as a prior, not as the vehicle's source of truth.
Waymo explicitly describes its HD maps this way: the map provides prior information that helps the system, particularly in complicated roads and poor visibility, while the vehicle's sensors remain responsible for what is actually happening now.
That gives you:
Map says:
"This intersection normally has three lanes and a left-turn pocket."
Sensors say:
"Construction has moved the lanes today."
Planner says:
"Trust current perception; deviate from the map."
That's much more robust than either pure map dependence or pretending maps have no value.
5. The real moat: the data engine
For a fleet operator, I would actually rank this above the neural-network architecture.
Build:
fleet → interesting events → automatic labeling → simulation → training → validation → OTA → fleet
You want the system continuously harvesting:
disengagement-like events
near collisions
unusual pedestrian behavior
construction
emergency vehicles
weird road geometry
adverse weather
sensor failures
planner uncertainty
novel objects
situations where humans behave unexpectedly
Then replay those situations in simulation thousands or millions of times before shipping a new model.
NVIDIA's current platform explicitly incorporates this concept through its AV Data Factory and simulation/training pipeline.
If I were building a commercial robotaxi fleet in 2026, I would not bet on a pure “end-to-end AI” stack or a camera-only architecture. I’d build a redundant, multi-sensor system with learned perception/prediction and a deterministic safety envelope, broadly following the architectural lessons demonstrated by Waymo and Mobileye.
The vehicle needs a safe state even when the primary autonomy computer fails
Maps
HD semantic map + continuously updated road-change layer
Useful, but don't make the vehicle dependent on centimeter-perfect maps
Fleet cloud
Data engine → simulation → training → validation → OTA deployment
The fleet should continuously improve without compromising release discipline
Remote assistance
Human-assisted exception handling, not remote driving as the normal fallback
Humans can resolve ambiguous situations without becoming a hidden driver
Sensor choice: lidar + camera + radar
This is the biggest architectural choice I'd make.
Waymo's current approach uses all three modalities—its sixth-generation system has 13 cameras, 4 lidar units and 6 radars—specifically to obtain overlapping, complementary views and redundancy. Waymo Its published architecture describes lidar for 3D geometry, cameras for visual semantics and radar for range/velocity and difficult weather.
Mobileye is pursuing a similar redundancy philosophy: its Drive system has independent camera and radar/lidar perception paths, alongside its RSS safety model.
I therefore wouldn't make camera-only sensing the foundation of a safety-critical commercial fleet, even though it can dramatically reduce hardware cost. Tesla's approach is the interesting counterexample, but I would rather optimize sensor cost after demonstrating the required safety envelope.
The software architecture
I'd structure the autonomy stack roughly like this:
The important detail is that the neural network should not be the final authority over safety.
A large multimodal model can propose that “the pedestrian will probably stay on the sidewalk.” The safety layer should still ask: What happens if that prediction is wrong?
What I'd buy vs. build
For a startup, I would buy the commodity layers and build the autonomy intelligence.
Buy/license:
Automotive compute platform
Cameras, radar and lidar
Vehicle platform
Drive-by-wire interface
GNSS/INS
Automotive Ethernet
Functional-safety infrastructure
Fleet charging/depot hardware
Build:
Sensor fusion
World model
Prediction
Behavior planning
Safety architecture
Simulation/data engine
Fleet learning pipeline
ODD management
Remote-assistance software
Validation and release infrastructure
I'd seriously evaluate NVIDIA's automotive compute ecosystem as the initial hardware platform rather than designing silicon immediately. Once the fleet reaches substantial scale, custom silicon becomes much more attractive.
Waymo is an instructive example: its newest onboard computer reportedly uses a custom 5-nm ASIC alongside components from suppliers including NVIDIA and AMD, with redundancy and ruggedness being explicit design priorities.
The key differentiator: the data engine
The real moat isn't the lidar.
It's:
fleet → data → scenario mining → simulation → training → validation → deployment → fleet
You want every interesting event to automatically become a regression test.
For example:
Car encounters unusual construction → logs scenario → automatically extracts scenario → generates 10,000 variations in simulation → new model trained → safety suite evaluates it → only then does the software reach the fleet.
That's considerably more valuable than simply collecting billions of miles.
And I'd maintain an enormous long-tail scenario library: jaywalking children, emergency vehicles, temporary lane markings, police directing traffic, flooded roads, unprotected left turns, cyclists behaving unpredictably, fallen objects, construction workers, unusual vehicles, etc.
Recent real-world experience reinforces why this matters: Waymo has had to update its fleet software following incidents involving flooding and unusual roadway behavior in San Antonio.
What about end-to-end driving models?
I'd absolutely use them—but inside a larger safety architecture.
A strong 2026 architecture could look like:
multimodal foundation model → latent/world representation → trajectory proposals → safety-constrained planner → conventional control
rather than:
cameras → giant neural network → steering wheel
The latter is elegant but gives you fewer independent mechanisms for detecting model failure.
Waymo itself has recently argued that there isn't a simple AI shortcut to fully autonomous driving; the challenge is the complete safety, sensing, compute, validation and operational system.
If I had to pick one reference architecture
I'd target something approximately like:
Waymo-style sensing + Mobileye-style formal safety concepts + modern multimodal foundation models + NVIDIA-class compute + an extremely aggressive fleet data/simulation engine.
That is, in my view, a stronger starting point than trying to reproduce either Waymo or Tesla exactly.
The business architecture matters just as much as the driving stack: start with a tightly bounded ODD (specific cities, weather, road classes and operating hours), prove safety there, then expand the ODD. Don't attempt “drive anywhere” on day one.
If you're actually designing this as a company, I can also lay out a concrete BOM and software architecture for a $20k, $30k, or $50k robotaxi, including sensor models, compute, networking, redundancy, cloud infrastructure and approximate per-vehicle economics.
When evaluating the "best" self-driving technology stack for a commercial robotaxi fleet, the industry has largely converged on an architecture that balances multimodal sensor redundancy with high-performance centralized compute and rigorous operational scaling.
There is no single off-the-shelf "best" stack, but the gold standard follows a proven modular and hardware-accelerated architecture.
1. The Core Technology Stack Architecture
The Sensor Suite (Multimodal Redundancy):
Industry leaders (like Waymo and Zoox ) rely on a heavy sensor mix combining LiDAR (for precise depth and geometry), long/short-range Radar (for velocity and weather penetration), and high-resolution Cameras (for semantic understanding and traffic light/sign detection). While vision-only stacks are leaner, a multi-sensor fusion approach remains the industry benchmark for commercial Level 4 safety guarantees.
Compute & AI Platform:
Real-time data ingestion requires massive parallel processing. The industry standard compute backbone relies heavily on high-performance automotive-grade GPUs and centralized architectures—such as the NVIDIA DRIVE platform (including AGX Thor and Hyperion architectures)—which process data streams equivalent to multiple streaming libraries every second.
Perception, Prediction, and Planning Software:
The software stack uses deep learning for object classification and tracking, predicting the future trajectories of pedestrians and other vehicles, and real-time motion planning to navigate complex urban environments safely.
Fleet Operations & Remote Assistance Layer:
Crucial to a robotaxi stack is the infrastructure outside the car. This includes cloud telemetry, predictive maintenance, fleet uptime management, and a remote-guidance center capable of assisting vehicles when they encounter edge-case blockages (construction, unmapped detours, or police interactions).
2. Leading Philosophies in the Market
The Multimodal / High-Redundancy Approach (e.g., Waymo, Zoox, Mobileye):
Pros: Widely regarded as the safest and most legally/operationally proven path to unconstrained Level 4 driverless deployment in dense urban cores. Backed by millions of commercial rider miles.
Cons: Higher bill-of-materials (BOM) cost per vehicle due to expensive sensor suites (LiDAR, radar).
The Vertically Integrated Vision-First Approach (e.g., Tesla):
Pros: Significantly lower hardware cost, highly scalable manufacturing potential, and end-to-end neural network training on massive consumer fleet data.
Cons: Faces steeper regulatory hurdles for true driverless (unattended) commercial robotaxi deployment without a safety driver in various complex jurisdictions compared to mature L4 competitors.
The Verdict
For a viable commercial robotaxi fleet, the best stack is a turnkey, multi-sensor Level 4 system (such as Mobileye Drive or an NVIDIA DRIVE -powered custom stack) paired with a purpose-built vehicle. Relying purely on vision works for driver-assist frameworks, but true uncrewed commercial scaling heavily favors heavy sensor redundancy and robust remote operations support.
Are you looking to build a stack from scratch , integrate an existing turnkey solution (like Mobileye or NVIDIA) , or analyze a specific competitor's business model?
There is no single "best" self-driving technology stack , as the industry is split between two fundamentally different engineering and economic philosophies: the Sensor-Dense/HD-Map Stack (pioneered by Waymo and Baidu Apollo Go ) and the Vision-Only/End-to-End Neural Network Stack (pioneered by Tesla).
Choosing the "best" stack depends on whether your priority is immediate, zero-safety-driver commercial scaling in mapped geo-fences or infinitely scalable, lower-cost deployment anywhere.
1. The Sensor-Dense + HD-Map Stack (The "Waymo/Baidu" Approach)
This architecture relies on overlapping redundancies and hyper-precise pre-mapping to minimize the burden on real-time edge-case AI.
Core Components: A heavy suite of LiDAR, RADAR, long/short-range cameras, and ultrasonic sensors paired with centimeter-accurate High-Definition (HD) maps.
How it Works: The vehicle uses real-time sensor data to cross-reference an existing 3D map of the city. Localization is trivial because the car already knows where every curb, sign, and lane line is. Planning and perception treat anomalies as exceptions to the map.
Pros:
Exceptional safety and regulatory approval track record for driverless commercial deployment in complex urban environments.
Highly deterministic; easier to prove safety cases to local regulators.
Cons:
Extremely expensive hardware suite (LiDAR units historically cost thousands of dollars).
Scaling is bottlenecked by the labor-intensive process of HD-mapping and validating new cities. If a road layout changes, the map must be updated.
2. The Vision-Only / End-to-End AI Stack (The "Tesla" Approach)
This architecture mimics human driving by feeding raw video streams directly into a massive neural network that outputs steering, braking, and acceleration commands.
Core Components: An array of high-resolution surround cameras (no LiDAR or RADAR), backed by heavy onboard inference computers (like Tesla's custom HW4/AI hardware) and vast cloud-based training compute clusters.
How it Works: Rather than using hard-coded rules or pre-existing HD maps, the AI learns vector space perception and driving policy directly from billions of miles of driving data.
Pros:
Massively scalable and cost-effective; hardware costs per vehicle are a fraction of sensor-dense builds, allowing for a much higher gross margin on fleet economics.
Operates anywhere roads exist without needing pre-existing HD maps.
Cons:
"Black box" problem—harder to mathematically prove why the neural network made a specific decision during a rare edge case.
Requires immense, continuous data loops and compute infrastructure to train out hallucinations or rare corner cases.
Key Layers of a Modern Robotaxi Stack
Whichever philosophy you lean toward, an enterprise-grade stack requires four integrated sub-systems:
Perception Layer: Responsible for object detection, classification, and tracking (pedestrians, cyclists, debris). Sensor-dense uses sensor fusion (LiDAR+Camera); vision-only relies strictly on spatial video transformers.
Localization & Mapping: Determining exact centimeter-level location. Sensor stacks use HD map matching + wheel odometry/IMU; vision stacks use visual odometry and real-time neural mapping.
Planning & Control (Behavioral & Trajectory): The "brain" that decides whether to go around a parked truck, yield, or stop. Modern leading stacks are shifting toward End-to-End (E2E) imitation learning here, replacing rigid behavior trees with neural networks trained on human driving trajectories.
Teleoperation & Fleet Management: No robotaxi stack is 100% autonomous out of the box. A robust remote assistance backend—capable of giving path-approval or nudging a stuck vehicle around construction zones—is mandatory for commercial viability.
The Verdict
If you are launching a commercial fleet tomorrow in dense, regulated metro areas , a Sensor-Dense + HD-Map framework gives you the highest short-term regulatory certainty, despite higher vehicle capital expenditures.
If you are playing a long-term game for global volume and unit economics , a Vision-Only, End-to-End Neural Network stack is structurally cheaper and infinitely more scalable, provided your AI training pipeline can clear the safety threshold.
To help narrow down what fits your specific business model, tell me:
What is your target deployment environment (dense urban centers vs. suburban/rural sprawl)?
Are you aiming to manufacture your own purpose-built vehicles (like a pod) or retrofit existing passenger cars?
If I were building a commercial L4 robotaxi fleet in 2026, I’d use a Waymo-style multi-sensor architecture, but build it around an NVIDIA DRIVE Hyperion-class compute platform rather than trying to invent the entire hardware stack from scratch.
My recommended stack
Layer
Recommendation
Why
Sensors
LiDAR + imaging radar + cameras + ultrasonics
Redundancy across weather, lighting and object types
AI flexibility without letting an unconstrained model control the vehicle
Control
Model-predictive / optimized vehicle control with independent safety monitors
Stable, predictable actuation
Maps
Continuously updated HD/semantic maps
Construction, lane changes, signals and road topology
Simulation
Massive closed-loop simulation + replay of real-world edge cases
Critical for long-tail validation
Fleet data
Automated data mining → labeling → training → simulation → OTA
The fleet should continuously improve itself
Remote assistance
Human-in-the-loop exception handling, not remote driving
Scales better and avoids turning operators into virtual drivers
Cybersecurity
Secure boot, hardware root of trust, signed OTA, intrusion detection
Essential for a connected driverless fleet
NVIDIA's current DRIVE Hyperion 10 is unusually close to this architecture: NVIDIA specifies dual Thor SoCs, 14 cameras, 9 radars, 1 lidar, 12 ultrasonics and redundant safety architecture, with ASIL-D capability.
The key design decision: don't go camera-only
For a robotaxi, I'd choose camera + lidar + radar.
Waymo's current system explicitly combines all three; its cameras provide visual semantics, lidar supplies detailed 3D geometry, and radar contributes velocity and robustness in rain, fog and snow.
That's particularly important because a robotaxi doesn't have a human sitting behind the wheel as a fallback. A sensor failure or degraded visibility therefore needs to be handled as an engineering problem, not a driver-assistance problem.
Tesla's camera-only approach is a fascinating counterexample, but I wouldn't choose it for a new fleet whose primary objective is maximum safety margin and predictable L4 deployment. Recent robotaxi deployments continue to show the industry converging on heterogeneous sensing rather than relying exclusively on vision.
Where I'd differentiate
I wouldn't try to beat Waymo by merely buying better sensors. The moat should be the data + autonomy software + validation loop.
The architecture I'd aim for is:
Sensors → multimodal foundation model → world model → prediction → behavior planning → trajectory planning → safety supervisor → controls
with a parallel path:
Fleet → event mining → labeling → training → simulation → safety validation → OTA deployment
That's essentially the direction the leading stacks are moving. NVIDIA, for example, now describes its platform as spanning data/model development, simulation, validation and vehicle compute, while Waymo describes a tightly co-designed hardware/ML system and recently disclosed custom silicon for processing lidar, radar and camera streams.
One thing I'd not outsource
I would buy the compute platform, sensors and vehicle, but keep these in-house:
Perception/world modeling
Prediction
Behavior planning
Safety architecture
Fleet data engine
Simulation/validation
Mapping/localization
Remote-assistance software
OTA deployment infrastructure
Those are the components that determine whether you actually have a robotaxi business or simply a vehicle equipped with somebody else's autonomy kit.
The closest real-world architectures
Waymo is the benchmark I'd study for the complete system. Its architecture combines custom sensors, multimodal perception, mapping, localization, prediction and planning, and it has accumulated enormous real-world autonomous driving experience.
Zoox is particularly interesting if you're designing the vehicle itself rather than retrofitting an existing car. Its mapping/localization architecture combines lidar, cameras, IMU and vehicle signals, with fleet-wide map updating.
NVIDIA DRIVE Hyperion is probably the most interesting commercial foundation if you're a new OEM/robotaxi operator and don't want to build the entire automotive compute/safety stack yourself. NVIDIA is explicitly positioning it as a production L4 robotaxi reference architecture.
Bottom line: I'd build a Waymo-inspired autonomy stack on NVIDIA Hyperion/Thor-class hardware, with heterogeneous sensing, multimodal AI, a deterministic safety envelope, continuously learned fleet intelligence, and extremely aggressive simulation/validation.
That gives you a much more credible path to L4 than trying to assemble a "camera + LLM + GPS" system and hoping scale will solve the edge cases.
I wouldn't try to beat Waymo by merely buying better sensors. The moat should be the data + autonomy software + validation loop.
The architecture I'd aim for is:
Sensors → multimodal foundation model → world model → prediction → behavior planning → trajectory planning → safety supervisor → controls
with a parallel path:
Fleet → event mining → labeling → training → simulation → safety validation → OTA deployment
That's essentially the direction the leading stacks are moving. NVIDIA, for example, now describes its platform as spanning data/model development, simulation, validation and vehicle compute, while Waymo describes a tightly co-designed hardware/ML system and recently disclosed custom silicon for processing lidar, radar and camera streams.
One thing I'd not outsource
I would buy the compute platform, sensors and vehicle, but keep these in-house:
The best self-driving technology stack for a commercial robotaxi fleet depends on a fundamental trade-off between high-redundancy multi-sensor arrays (LiDAR/Radar/Camera) optimized for safety and geofenced scaling, versus vision-only neural networks optimized for mass-manufacturing cost and generalized scaling.
The industry is split between two primary architectural philosophies, each defining a different approach to the autonomous stack.
The Multi-Sensor Hybrid Stack (e.g., Waymo, Baidu, Zoox)
This architecture relies on overlapping modalities to ensure safety-critical redundancy, allowing operation without a human safety driver within strictly mapped urban domains.
Sensor Suite: Combines long-range and short-range LiDAR , 360-degree cameras, radar , and ultrasonic sensors. LiDAR provides active, precise depth and geometry mapping independent of lighting conditions.
Compute & Software: Heavy reliance on high-definition (HD) mapping combined with real-time localization and modular software pipelines (perception, prediction, planning, and control).
Trade-offs: High hardware and vehicle-retrofitting costs ($70,000–$130,000+ per vehicle), making dense geographic expansion capital-intensive, but yields proven commercial driverless reliability in complex urban cores.
The Vision-Only End-to-End Neural Stack (e.g., Tesla)
This architecture mimics human driving biology by relying exclusively on high-resolution camera inputs processed through deep learning neural networks.
Sensor Suite: Purely surround-camera arrays coupled with powerful onboard AI inference computers (such as Tesla's HW4/FSD hardware).
Compute & Software: End-to-end neural networks where video inputs map directly to steering, braking, and acceleration controls, trained on massive fleet-scale driving video data rather than manually labeled HD maps.
Trade-offs: Significantly lower hardware production costs ($30,000–$35,000 per vehicle) and massive theoretical scalability, but historically faces higher regulatory hurdles and edge-case validation challenges in unmapped or adverse weather environments.
Core Technology Stack Comparison
Metric / Attribute
Multi-Sensor Hybrid (LiDAR + Camera)
Vision-Only (Camera-Centric AI)
Primary Advantage
High geometric safety redundancy and reliable immediate L4 deployment.
Exceptional manufacturing cost efficiency and mass scalability.
Hardware Cost
High ($70k–$130k+ per vehicle integration).
Low (<$35k per vehicle at scale).
Mapping Dependency
Relies heavily on pre-existing High-Definition (HD) maps.
Operates dynamically without mandatory HD map pre-scans.
If you want to narrow this down further, tell me:
Are you planning for unconstrained general deployment or a geofenced urban market?
Is your primary constraint capital expenditure per vehicle or speed of regulatory approval?
If I were building a new commercial robotaxi fleet in 2026, I’d use a Waymo-style multimodal architecture, but build it on NVIDIA DRIVE Hyperion rather than trying to reproduce Waymo’s proprietary stack.
The key is not simply “the best AI model.” For robotaxis, the winning stack is the one that combines sensor redundancy, deterministic real-time compute, strong simulation/data infrastructure, fail-operational safety, and fleet operations.
NVIDIA's current Hyperion platform is unusually attractive for this because it explicitly packages the compute, sensor architecture and AV software around L4 development. Its current Hyperion 10 specification calls for two DRIVE AGX Thor SoCs, with cameras, radar, lidar, ultrasonics and interior sensing, and NVIDIA says the platform targets ASIL-D-capable safety architecture.
The sensor philosophy I'd choose
I would not build a camera-only robotaxi.
Waymo's current production approach is instructive: its sixth-generation Driver combines high-resolution cameras, imaging radar and lidar, explicitly using different modalities for redundancy. Waymo Waymo's publicly described architecture similarly uses lidar for 3D geometry, cameras for semantic understanding and radar for range/velocity and adverse-weather robustness.
For a robotaxi, I'd rather spend an extra few thousand dollars on sensing than discover that a particular combination of glare + rain + dirty camera + unusual pedestrian behavior defeats the system.
Where I'd spend the engineering budget
The biggest mistake would be spending 80% of the effort on the neural network.
I'd roughly prioritize:
Data engine — 20–25%
Perception/prediction/planning — 20–25%
Safety architecture and validation — 15–20%
Simulation and scenario generation — 10–15%
Vehicle integration, controls and compute — 10–15%
Fleet/cloud/remote operations — 5–10%
The fundamental competitive advantage becomes the closed loop:
fleet → difficult events → data selection → labeling → training → simulation → safety validation → OTA deployment → fleet
That's essentially what the mature players are converging toward. NVIDIA describes its own architecture as a data-to-model-to-simulation-to-vehicle loop, while Waymo is increasingly co-designing its silicon, sensors and ML stack to reduce latency and power consumption.
What I would not do
I wouldn't build a giant HD-map-dependent system. Use maps as a powerful prior, but make the vehicle capable of handling temporary map inaccuracies and unmapped situations.
I wouldn't rely entirely on an end-to-end neural network. Learned driving models are extremely promising, but a commercial L4 system needs independent safety mechanisms around them.
I wouldn't make remote operators part of the normal control loop. Remote assistance should resolve exceptional uncertainty, while the vehicle remains responsible for actually driving.
I wouldn't optimize sensor cost before achieving safety. Once the system has demonstrated safety, you can aggressively reduce sensor count and cost. Waymo's sixth-generation system is an example of this trajectory: it claims substantially reduced sensing hardware while maintaining multimodal redundancy.
The interesting strategic choice
There are really three architectures competing now:
NVIDIA ecosystem: standardized L4 hardware/software platform that lets multiple vehicle and AV companies build on a common architecture.
Wayve/Tesla-style learned vision-first approaches: substantially more neural/end-to-end and less dependent on lidar and conventional HD maps. Wayve's current AV2.0 approach, for example, emphasizes a learned AI driver without traditional HD maps or lidar.
For a new entrant whose goal is actually operating thousands of robotaxis, I'd choose #2 as the foundation and borrow heavily from #1's safety philosophy.
In other words:
NVIDIA Hyperion hardware + multimodal sensing + your own learned perception/prediction/planning stack + independent safety layer + massive simulation/data engine.
That gives you a much more credible path to production than trying to invent the entire vehicle compute architecture yourself, while still leaving the part that creates long-term differentiation—the driving intelligence and fleet data flywheel—under your control. NVIDIA is also rapidly expanding this ecosystem, with automakers and mobility companies adopting Hyperion for L4 programs in 2026.
If you're evaluating this from an investment/startup architecture perspective, I can also lay out a concrete $10M / $50M / $200M robotaxi technology roadmap, including recommended sensors, compute BOM, software components, team size, and what I'd build vs. buy.
The biggest mistake would be spending 80% of the effort on the neural network.
