Custom AI chip design
Microsoft and Google Cloud TPU are nearly even for custom AI silicon design and NPU evaluation.
It depends
Broadcom’s answer sentences instead focus on custom-chip co-design for infrastructure and AI workloads.
Since Jul 5Updated Nov 14, 2026Based on 63 AI answers
Reviewed by Dimitry Apollonsky
Brands AI mentions
Its Maia accelerators target high-throughput AI and data-center workloads.
Its custom-chip work spans infrastructure partnerships and AI processor co-design.
Its MTIA accelerators address internal workloads, including recommendation-model inference.
Also mentioned rarely: Cadence · cadence.com, Arm, Synopsys, Tenstorrent, Alchip Technologies, Amazon, OpenAI, Samsung, MediaTek, Amazon Web Services, Global Unichip Corporation, Meta AI, SambaNova, Huawei, Google Search, Google Store, Etched, Snapdragon, Socionext, Graphcore, Intel Foundry, Samsung Foundry, Tesla, Anthropic, Microsoft Azure, Dojo, Global Unichip Corp, Annapurna Labs, CEVA, Hexagon, NVIDIA Inference (NVIDIA Inference Stack), NXP Semiconductors, PyTorch, Apache TVM, MTIA, Amazonaws, Apple Neural Engine, Coral, d-Matrix, Siemens EDA, GlobalFoundries, Kirin Holdings, MLIR, Samsung Exynos, VeriSilicon, Alibaba Group, AMD Ryzen, Arm Ethos-U, Arm Neoverse, Microsoft Azure Maia, CoWoS, Cadence · getcadence.app, Imagination Technologies, Instinct MI, Intel Core, JAX, M-series, NVIDIA Blackwell, Rebellions, Samsung Exynos NPU, Tesla Universal Wall Connector, Triton, TSMC 3DFabric, Alphabet, AI Engine, Altera, AMD Instinct, AMD Xilinx, AMD Vitis, Ryzen AI NPU, AMD Vivado Design Suite, AMD Ryzen AI, Amkor Technology, American Public University System, Architect Labs, ASE Technology, ASML, Meta · atmeta.com, AWS Graviton, Azure Cobalt, Cadence Innovus, Cadence Palladium, JasperGold, Tensilica, Cambricon, Google Cloud, NVIDIA TensorRT, Coral NPU, EdgeSilica, ESP (Embedded Systems Platform), Faraday Technology, GAIA PC NPU, gem5, Hailo, Imagination, Intel Gaudi, Ironwood, Llvm, MosChip Technologies, Neuchips, ONNX, Quadric, Qualcomm Snapdragon, Qualcomm Hexagon, Questa Capital, SEMIFIVE, Siemens Calibre, SiFive, Silicon Patterns, Sohu, Synaptics, Synopsys ZeBu, Synopsys Fusion Compiler, VC Formal, SystemC, Google Gemini, Tesla Full Self-Driving, Utmel, Vera Rubin, Verified Carbon Standard (VCS), Xcelium
- MicrosoftChatGPTGoogle AI Mode
- Google Cloud TPUChatGPTGoogle AI Mode
- AWS InferentiaChatGPTGoogle AI Mode
- BroadcomChatGPTGoogle AI Mode
- MetaChatGPTGoogle AI Mode
Questions and answers
ChatGPTNov 7, 2026
- 1Broadcom
- 2Marvell
- 3Alchip Technologies
- 4MediaTek
- 5VeriSilicon
- 6Socionext
- 7Faraday Technology
- 8Google
- 9Amazon
- 10Microsoft
- 11MAIA
- 12Meta
- 13MTIA
- 14Apple
- 15Qualcomm
- 16Huawei
- 17Cambricon
- 18Arm
- 19Synopsys
- 20Cadence
- 21Siemens EDA
- 22CEVA
- 23Samsung Foundry
- 24ASE Technology
- 25Global Unichip Corporation
- 26Google Cloud TPU
- 27AWS
- 28AWS Inferentia
- 29Arm Neoverse
- 30Taiwan Semiconductor Manufacturing Company (TSMC)
The custom AI silicon market includes two distinct groups: companies that design complete custom NPUs/AI ASICs and engineering partners that help customers turn an architecture into manufacturable silicon.
For an NPU evaluation, I would focus first on the second group: ASIC design partners, independent design houses, and silicon IP providers.
1. Leading custom AI silicon designers
PRIORITY COMPANIES TO EVALUATE
Company Headquarters Core capabilities Broadcom US Custom AI ASICs, XPU co-design, high-speed interconnect Marvell US Custom AI accelerators, networking and memory interfaces Alchip Technologies Taiwan Advanced-node ASIC design and physical implementation Global Unichip (GUC) Taiwan Custom ASIC design, HBM integration, advanced packaging MediaTek Taiwan Custom SoCs, AI accelerators and edge AI VeriSilicon China NPU IP, AI ASIC architecture and turnkey design services Socionext Japan Custom SoCs and ASIC development Faraday Technology Taiwan ASIC design services and semiconductor IP Broadcom and Marvell are the two particularly important players in hyperscale custom AI silicon. Alchip and GUC are important alternatives for outsourced ASIC engineering.
2. Companies designing their own AI processors
These are potential technology competitors, partners, or benchmarks.
Company AI silicon platform Google TPU Amazon Trainium and Inferentia Microsoft Maia Meta MTIA Apple Neural Engine Qualcomm Hexagon NPU Huawei Ascend Cambricon MLU AI processors These companies have varying degrees of internal design capability and external design-partner involvement.
3. Silicon IP and design-tool providers
These companies may be especially relevant if you are developing an NPU rather than purchasing a finished accelerator.
Company What it provides Arm CPU architecture, compute IP and Neoverse Synopsys Design IP, RTL tools, verification and implementation Cadence EDA tools, simulation, verification and chip design Siemens EDA Physical verification and semiconductor design tools CEVA DSP and AI processor IP VeriSilicon Proprietary NPU and AI accelerator IP www.verisilicon.comchipsupplychain.orgarmkeil.blob.core.windows.net
4. Manufacturing partners
Don't overlook the manufacturing ecosystem:
TSMC
Leading advanced-node foundry and advanced packaging provider.
Samsung Foundry
Advanced-node manufacturing and packaging.
ASE Technology
Advanced packaging, assembly and testing.
5. My recommended shortlist for NPU evaluation
I would prioritize these six companies:
Company Why evaluate Broadcom Large-scale AI ASIC co-design Marvell Custom accelerator engineering Alchip ASIC implementation expertise GUC Advanced-node and HBM integration VeriSilicon NPU IP and end-to-end design Socionext Custom SoC engineering An important distinction: Broadcom, Marvell and the design houses are potential engineering partners. Google, Amazon and Microsoft are examples of organizations developing proprietary AI silicon. These are different business models.
For your evaluation, the next important question is whether you need an NPU architecture developed from scratch, licensed NPU IP integrated into your SoC, or a complete turnkey ASIC design.
I can also prepare a competitive landscape comparing these companies by NPU architecture, design capabilities, licensing model, process nodes and suitability for custom NPU development.
Compare the six leading NPU design companies Identify turnkey NPU design partners
Pages cited in this answer7
- thebuildout.aiCustom AI Silicon & Hyperscaler ASICs: AI infrastructure company list | The Buildout
- sotkn.comWho Makes Custom AI Chips? AI ASIC Partners (2026)
- dataintelo.comASIC Design Service Market Size, Share & Forecast Report 2025 to 2034 | Dataintelo
- aimultiple.comTop 30+ AI Chip Makers: NVIDIA & Its Competitors
- verisilicon.comAbout VeriSilicon
- chipsupplychain.orgWho makes what in the AI chip supply chain · Chip Supply Chain
- armkeil.blob.core.windows.netCHAPTER 1 SIGNPOSTS ON THE ROAD TO AI COMPUTING
Google AI ModeNov 7, 2026
- 1NVIDIA
- 2Vera Rubin
- 3AMD
- 4Instinct MI
- 5Intel
- 6Google
- 7Amazon
- 8AWS
- 9Microsoft
- 10MAIA
- 11Meta
- 12MTIA
- 13Broadcom
- 14Marvell
- 15Tenstorrent
- 16SambaNova
- 17Etched
- 18d-Matrix
- 19Rebellions
- 20Samsung
- 21Intel Foundry
- 22NVIDIA Blackwell
- 23AMD Ryzen
- 24Intel Core
- 25Google Cloud TPU
- 26AWS Inferentia
- 27Cerebras Systems
- 28Taiwan Semiconductor Manufacturing Company (TSMC)
Custom AI silicon and neural processing unit (NPU) design is split across
major merchant vendors, hyperscale cloud giants (often partnering with ASIC specialists), and specialized startups.Market analysis from AI Windes and AIMultiple outlines the primary players designing custom AI and NPU silicon:
1. Merchant AI & NPU Designers (Sell Openly to the Market)
- Nvidia : Dominates data center and edge AI with its GPU and custom system architectures (such as Hopper, Blackwell, and the Vera Rubin platform).
- AMD : Designs merchant AI accelerators (like the Instinct MI series) and embeds XDNA-based NPUs into its Ryzen PC processors.
- Intel : Builds merchant data center accelerators (Gaudi) alongside client-side NPU silicon integrated directly into Core and Core Ultra client processors.
2. Hyperscalers (Design In-House / Custom Silicon)
- Google : Designs its own Tensor Processing Units (TPUs) and custom inference/training ASICs for internal workloads.
- Amazon (AWS) : Designs custom AI chips like Trainium (for training) and Inferentia (for inference).
- Microsoft : Develops custom silicon projects (such as the Maia accelerator series) to power its Azure cloud and internal AI models.
- Meta : Builds custom inference accelerators like the MTIA (Meta Training and Inference Accelerator) family.
3. Custom ASIC Co-Design Partners
- Broadcom : Acts as the primary silicon co-designer and execution partner for hyperscalers—co-developing Google's TPUs, Meta's MTIA, and custom chips for other major cloud and AI firms.
- Marvell : Provides custom ASIC design services and high-speed connectivity IP for cloud service providers building proprietary AI silicon.
4. Independent AI & NPU Startups
- Tenstorrent : Designs RISC-V based AI and NPU chiplets and hardware scaling solutions.
- Cerebras : Builds massive wafer-scale processors (WSE) tailored for high-speed AI compute.
- SambaNova Systems : Focuses on reconfigurable dataflow units (RDUs) for enterprise AI inference and training.
- Etched, d-Matrix, & Rebellions : Build domain-specific ASICs and rack-scale inference accelerators targeting lower-latency or transformer-specific workloads.
(Note: The vast majority of these designers do not own fabrication plants; instead, they outsource physical manufacturing almost exclusively to TSMC , with smaller volumes handled by Samsung or Intel Foundry.)
Are you evaluating NPUs for edge/client devices (like AI PCs or phones) or server/data center workloads , and do you have a specific target performance or power envelope in mind?
ChatGPTNov 1, 2026
- 1AMD
- 2Samsung Foundry
- 3GlobalFoundries
- 4PyTorch
- 5ONNX
- 6Xcelium
- 7Synopsys
- 8Cadence
- 9Cadence Innovus
- 10Synopsys Fusion Compiler
- 11JasperGold
- 12VC Formal
- 13Siemens Calibre
- 14Llvm
- 15ESP (Embedded Systems Platform)
- 16AMD Vitis
- 17Taiwan Semiconductor Manufacturing Company (TSMC)
- 18Questa Capital
- 19Verified Carbon Standard (VCS)
- 20AMD Vivado Design Suite
- 21Altera
How to design, develop, and manufacture custom AI accelerator silicon
Creating a custom silicon accelerator for specialized AI workloads is a multidisciplinary engineering process that combines AI architecture, computer engineering, semiconductor physics, chip design, software development, and advanced manufacturing.
The goal is to build a processor whose hardware is specifically optimized for a particular class of AI computations, rather than relying on a general-purpose CPU or GPU.
For example, imagine you want to develop a custom AI chip for:
- Large language model (LLM) inference.
- Transformer-based neural networks.
- Computer vision and image recognition.
- Autonomous robotics.
- Speech recognition and real-time audio processing.
- Scientific AI and simulation.
- Edge AI devices with extremely low power consumption.
The process is broadly similar across these applications, although the architecture, memory system, precision requirements, and manufacturing choices can differ substantially.
I'll walk through the process from the initial idea through architecture design, RTL development, verification, fabrication, packaging, software, and commercial production.
1. Define the AI workload before designing the chip
(Starting point)
The biggest mistake in custom silicon development is starting with the physical chip design before understanding the computational workload.
You first need to identify exactly what the accelerator will do.
Consider three possible products:
Accelerator Primary workload Important design priorities LLM inference ASIC Transformer inference Memory bandwidth, matrix multiplication, latency Computer vision ASIC CNNs and vision transformers Parallel processing, image throughput, power Edge AI ASIC Small neural networks Low power, small die, low cost You should characterize the AI models you intend to support.
For example, with an LLM accelerator, you would examine:
- Model parameter count.
- Number of transformer layers.
- Attention head dimensions.
- Sequence length.
- Batch size.
- KV-cache requirements.
- Weight storage requirements.
- Quantization requirements.
- Target tokens per second.
- Maximum acceptable inference latency.
Translate AI operations into hardware requirements
Neural networks perform large numbers of mathematical operations.
Some of the most important are:
- Matrix multiplication.
- Vector multiplication.
- Convolution.
- Attention calculations.
- Normalization.
- Activation functions.
- Reduction operations.
For example, matrix multiplication:
mathC=A\times Bis one of the fundamental operations in modern AI.
A specialized accelerator can implement thousands of multiplication-and-accumulation operations in parallel.
You should establish measurable design targets:
Metric Example target Performance 100 TOPS Power consumption 50 W Memory bandwidth 1 TB/s Numerical precision INT8 / BF16 Manufacturing process 5 nm-class Application LLM inference These are illustrative targets, not specifications for a particular chip.
Deliverable: A workload specification describing what the chip must compute, how quickly it must compute it, how much energy it can consume, and what it will cost.
2. Select the accelerator architecture
This is where you decide what kind of processor you are actually building.
There are several major architectural approaches.
A. Systolic array architecture
A systolic array uses a grid of processing elements that exchange data in a coordinated pattern.
Each processing element performs arithmetic operations, typically multiply-accumulate operations.
Advantages:
- Excellent matrix multiplication efficiency.
- High computational density.
- Predictable data movement.
- Good energy efficiency for regular workloads.
This is particularly useful for transformer and neural-network matrix operations.
B. Vector processing architecture
Vector processors operate on multiple data elements simultaneously.
They are useful for:
- Element-wise operations.
- Activation functions.
- Normalization.
- Reductions.
- General tensor operations.
C. Spatial dataflow architecture
A spatial architecture distributes computations across multiple specialized hardware units.
Instead of repeatedly moving data between a centralized processor and memory, the design attempts to keep data close to where computation occurs.
This can significantly reduce energy consumption.
D. Hybrid architecture
For many specialized AI accelerators, a hybrid architecture makes the most sense.
For example:
Illustrative AI accelerator architecture
Host interface / command processor
PCIe or another system interface
Matrix engines
Tensor operations
Vector engines
Element-wise operations
On-chip SRAM
Fast local storage
Memory controller
External memory
High-bandwidth memory / external memory
Conceptual architecture; actual implementations vary.
The architecture should be driven by the workload rather than by a desire to include as many processing units as possible.
3. Design the memory subsystem
This is one of the most important parts of an AI accelerator.
In many AI workloads, the bottleneck isn't the speed of the arithmetic units. It's getting data to those units quickly enough.
For example, an LLM might have enough computational capacity to generate tokens quickly, but its performance can be limited by moving model weights and KV-cache data through memory.
You need to design several levels of memory.
Memory type Purpose Characteristics Registers Immediate calculations Extremely fast, very small SRAM Local tensor storage Fast, on-chip L1/L2 buffers Reusable data Low-latency access HBM Large model data Very high bandwidth External DRAM Additional capacity Larger but slower Important design decisions
On-chip SRAM: Determines how much data can remain close to the processing engines.
HBM: High-bandwidth memory is especially useful for data-center AI accelerators.
Memory controllers: Coordinate data movement between processing units and external memory.
Interconnect: Connects compute engines, memory controllers, and other subsystems.
Dataflow: Determines how tensors move through the processor.
For LLM inference, you must also consider:
- Weight streaming.
- KV-cache storage.
- Attention data movement.
- Prefetching.
- Memory allocation.
- Multiple concurrent inference requests.
A good accelerator architecture balances compute throughput with memory bandwidth and capacity.
4. Select numerical precision and implement AI-specific compute engines
Modern AI chips don't necessarily need to perform every operation using conventional 32-bit floating-point arithmetic.
Different AI workloads benefit from different numerical formats.
Format Typical application FP32 High-precision computation BF16 AI training and inference FP16 Neural-network computation FP8 Efficient large-model computation INT8 Quantized inference INT4 Highly compressed inference For example, an LLM inference accelerator might use:
- INT4 for quantized model weights.
- INT8 for selected intermediate computations.
- BF16 for operations requiring greater numerical range or precision.
The design team needs to establish acceptable numerical error and validate model accuracy under each supported format.
Develop specialized compute units
The main compute blocks might include:
- Matrix multiplication engines.
- Vector arithmetic units.
- Attention processing units.
- Normalization units.
- Activation-function units.
- Tensor movement engines.
- Reduction and accumulation units.
The objective is to implement frequently used AI operations efficiently in hardware.
An important architectural consideration is programmability: you don't want a chip that performs one particular model extremely well but cannot accommodate future models or software updates.
5. Develop the actual digital hardware (RTL design)
(Major engineering milestone)
Now you translate the architecture into hardware descriptions.
The primary technologies include:
- Verilog.
- SystemVerilog.
- VHDL.
- Chisel.
- SystemC.
- High-Level Synthesis (HLS).
RTL stands for Register Transfer Level.
It describes how digital hardware stores, transfers, and processes information over clock cycles.
For example, a simplified multiply-accumulate operation is:
mathACC_{next}=ACC+(A\times B)A real accelerator contains many such operations operating in parallel.
What engineers build
The RTL design is divided into manageable hardware modules.
Module Responsibility Compute engine Performs tensor operations Scheduler Distributes operations DMA engine Moves data Memory controller Coordinates memory Interconnect Connects hardware blocks Command processor Executes instructions Security controller Protects system resources PCIe interface Connects accelerator to host Engineers also integrate existing intellectual property where appropriate, such as processor cores, memory interfaces, PCIe controllers, and standard communication interfaces.
You generally don't want to reinvent every component.
Recommended development approach
Develop the architecture in progressively more detailed models:
- Python-based performance model.
- Cycle-accurate architectural simulator.
- Hardware-oriented C++ or SystemC model.
- RTL implementation.
- FPGA prototype.
This allows you to discover architectural mistakes before committing to expensive custom silicon.
A useful educational resource is Columbia University's ESP platform. Its documentation covers accelerator RTL development, integration, and high-level synthesis.
6. Verify and validate the hardware
This is one of the most expensive and time-consuming parts of semiconductor development.
You must establish that the design actually works before manufacturing it.
A silicon defect can cost millions of dollars to correct.
A. Functional verification
Engineers simulate the hardware and verify that it behaves according to its specifications.
They test:
- Arithmetic correctness.
- Memory access.
- Data movement.
- Instruction execution.
- Clock behavior.
- Reset behavior.
- Error handling.
- Multiple concurrent operations.
B. Formal verification
Formal verification uses mathematical techniques to establish that specified hardware properties hold across defined conditions.
It complements simulation, rather than replacing it.
C. Performance verification
You must establish that the chip can achieve its intended:
- Clock frequency.
- Compute throughput.
- Memory bandwidth.
- Latency.
- Power consumption.
D. FPGA prototyping
Before producing custom silicon, you can implement parts of the architecture on an FPGA.
An FPGA allows you to test real hardware behavior without commissioning a custom semiconductor fabrication run.
You can evaluate:
- Data movement.
- Hardware/software integration.
- Command execution.
- Driver functionality.
- Model execution.
- Architectural assumptions.
An FPGA will not necessarily reproduce the speed, power consumption, or memory characteristics of the eventual ASIC.
However, it is extremely useful for validating functionality.
AMD's Vitis documentation provides a practical example of the RTL-to-FPGA workflow, including simulation, synthesis, implementation, timing analysis, and application integration.
7. Synthesize the design and perform physical implementation
(From logical design to physical silicon)
Once the RTL is sufficiently verified, the design moves into physical implementation.
This is where your logical hardware becomes a physical arrangement of transistors and wires.
There are several major steps.
A. Logic synthesis
EDA software converts RTL into a gate-level netlist.
This involves mapping the design into actual logic cells available in the selected semiconductor manufacturing process.
The result is a representation of the hardware using gates, flip-flops, and other standard cells.
B. Floorplanning
Engineers determine where major blocks should physically reside on the chip.
For an AI accelerator, this includes:
- Matrix engines.
- SRAM blocks.
- Memory controllers.
- Interconnect.
- PCIe interfaces.
- Clock and power infrastructure.
The objective is to optimize physical area, power consumption, timing, and routing.
C. Placement and routing
The EDA software determines the physical locations of standard cells and creates the interconnections between them.
This is a difficult optimization problem because physical distance affects signal delay, power, congestion, and heat.
D. Timing analysis
Static timing analysis checks whether signals can propagate through the design within the required clock periods.
If the design cannot meet its timing requirements, engineers may need to modify the architecture, placement, or implementation.
E. Physical verification
Before fabrication, engineers conduct important sign-off checks, including:
- Design Rule Checking (DRC).
- Layout Versus Schematic (LVS).
- Static timing analysis.
- Power integrity analysis.
- Electromigration analysis.
- Signal integrity analysis.
- Thermal analysis.
A typical digital ASIC flow proceeds from RTL through synthesis, physical implementation, verification, and tapeout. CERN publishes examples of technology-specific RTL-to-GDS implementation flows.
8. Choose a semiconductor foundry and manufacturing process
Now you need to determine where and how your accelerator will be manufactured.
Most new AI hardware companies operate using a fabless model.
This means they design the chip but contract an external semiconductor foundry to manufacture it.
Some major manufacturing providers include:
TSMC
A major advanced semiconductor foundry serving fabless chip companies.
Samsung Foundry
Offers semiconductor manufacturing and advanced packaging capabilities.
GlobalFoundries
Provides specialty and mature-node semiconductor manufacturing.
Choosing the process node
You might encounter manufacturing processes described as:
- 180 nm.
- 65 nm.
- 28 nm.
- 16/14 nm.
- 7 nm.
- 5 nm.
- 3 nm.
Smaller process nodes can offer advantages in transistor density and potentially power and performance, but they also introduce greater design complexity and cost.
Importantly, a smaller node isn't automatically better for every accelerator.
For instance:
- A small edge inference chip might be economically attractive on a mature node.
- A high-performance data-center accelerator may justify an advanced process.
- A memory-intensive design may benefit more from packaging and memory architecture improvements than from a smaller transistor node.
The foundry design process
You need access to a Process Design Kit (PDK).
A PDK contains the technology-specific information necessary to design and verify a chip for a particular manufacturing process.
It includes information about:
- Standard-cell libraries.
- Transistor models.
- Design rules.
- Manufacturing constraints.
- Physical layout requirements.
- Electrical characteristics.
You must design within the foundry's manufacturing rules.
The foundry and chip-design team work together to ensure that the proposed design can be manufactured reliably.
9. Tapeout: Send the completed design for fabrication
Tapeout is the formal milestone at which the finalized chip layout is delivered for manufacturing.
The main physical design deliverable is typically a GDSII or OASIS layout database, accompanied by the required manufacturing and sign-off data.
The foundry uses this design to prepare the manufacturing process.
For a conventional monolithic digital ASIC, the manufacturing process broadly involves:
- Preparing silicon wafers.
- Depositing and processing material layers.
- Patterning structures using photolithography.
- Etching and doping selected regions.
- Building transistor structures.
- Creating metal interconnections.
- Performing wafer-level inspection and testing.
The exact process depends on the foundry and manufacturing technology.
Why tapeout is such a major milestone
Before tapeout, you can modify the digital design relatively easily.
After tapeout, correcting a fundamental silicon design defect may require another fabrication run.
That can mean significant additional engineering expense and schedule delays.
This is why extensive pre-silicon verification is essential.
10. Package the silicon and integrate high-bandwidth memory
For modern AI accelerators, manufacturing the silicon die is only part of the process.
You also need to package it.
Packaging provides:
- Electrical connections.
- Physical protection.
- Thermal management.
- Power delivery.
- External interfaces.
For high-performance AI chips, advanced packaging is particularly important.
Common packaging approaches
Packaging General use Conventional 2D Standard packaged chips 2.5D Multiple dies connected through an interposer or similar substrate 3D stacking Dies vertically stacked Chiplets Multiple specialized dies integrated into one package Why chiplets are interesting
Instead of manufacturing one enormous monolithic processor, you can divide functionality among multiple dies.
For example:
- Compute die.
- I/O die.
- Memory interface die.
- Specialized accelerator die.
These can be integrated into a single package.
The Universal Chiplet Interconnect Express (UCIe) standard is designed to provide standardized die-to-die connectivity and facilitate chiplet-based systems.
For an AI accelerator, a chiplet strategy can offer greater flexibility in combining compute, memory interfaces, and different manufacturing processes.
However, it adds packaging, interconnect, testing, and integration complexity.
11. Develop the software stack
(Essential for commercial success)
A custom AI accelerator is not a complete product just because the silicon works.
You need software that allows AI developers to actually use it.
This is one of the most underestimated aspects of building an AI chip.
A chip with excellent theoretical performance can struggle commercially if developers cannot easily run their models on it.
Your software stack should include several layers.
AI accelerator software stack
AI applications
LLMs, vision, speech, robotics
AI frameworks
PyTorch, ONNX and supported model interfaces
Model compiler and optimizer
Graph optimization, quantization, operator mapping
Runtime and libraries
Scheduling, memory management, kernel libraries
Device drivers
Hardware communication and resource management
Custom AI accelerator silicon
The major software components
1. Device drivers
These allow the operating system to communicate with the accelerator.
2. Runtime system
Manages hardware resources, memory allocation, execution queues, and synchronization.
3. Compiler
Converts AI model operations into instructions and kernels supported by your hardware.
4. Kernel libraries
Provide optimized implementations of matrix multiplication, attention, normalization, and other operations.
5. Framework integration
Enables developers to use your accelerator through familiar machine-learning frameworks and APIs.
6. Model optimization tools
Support quantization, graph transformations, operator fusion, and model compilation.
A particularly important decision: Build your own compiler or adapt an existing ecosystem?
For a first-generation accelerator, building an entirely new AI programming ecosystem is generally an expensive mistake.
A better strategy is often to support established model representations and frameworks, while developing a compiler backend and hardware-specific runtime.
For example, you could support ONNX model import and PyTorch-based workflows.
The software architecture should be planned alongside the hardware architecture, not after the silicon is completed.
12. First silicon bring-up and post-silicon validation
Once the manufactured chips return, you begin testing the actual physical silicon.
This is called post-silicon validation.
The first objective is to establish that the chip powers on and operates correctly.
Initial bring-up
Engineers typically work through:
- Inspecting the packaged devices.
- Installing the chip onto a test board.
- Checking power rails and electrical behavior.
- Establishing clock and reset functionality.
- Configuring the device.
- Testing host communication.
- Running basic memory tests.
- Executing simple compute operations.
- Running complete AI workloads.
What you measure
Measurement What it tells you Clock frequency Actual operating speed Power consumption Energy requirements Temperature Thermal behavior Memory bandwidth Data movement capability Compute throughput Actual AI performance Latency Response time Model accuracy Numerical correctness Stability Reliability under sustained operation You should compare actual silicon performance with your pre-silicon simulations.
Sometimes the first silicon revision reveals unexpected limitations:
- Timing problems.
- Memory bottlenecks.
- Excessive power consumption.
- Thermal issues.
- Software compatibility problems.
- Manufacturing defects.
Some problems can be addressed through firmware or software. Others require a hardware revision.
13. Manufacturing test, yield optimization, and mass production
A working prototype is not necessarily a commercially manufacturable product.
You need to establish that the chip can be manufactured consistently and economically.
A. Wafer testing
Before individual dies are packaged, manufacturers perform electrical testing to identify defective devices.
B. Dicing
The wafer is separated into individual dies.
C. Packaging
The dies are assembled into their final packages.
D. Final testing
The packaged devices undergo electrical, functional, and performance testing.
E. Reliability testing
Depending on the application, qualification can include:
- Temperature cycling.
- High-temperature operating life.
- Thermal stress.
- Electrical stress.
- Package reliability.
- Long-duration operation.
F. Yield optimization
Yield represents the proportion of manufactured devices that meet the required specifications.
For example, if a wafer produces 1,000 dies and 800 meet the required criteria, the initial yield is 80%.
Yield has a major influence on chip economics.
You must work with the foundry and packaging partners to improve manufacturing consistency, identify defects, and establish acceptable production specifications.
14. Build the actual AI accelerator product
The final chip is typically only one component of the product.
For a data-center accelerator, you might need:
- Custom accelerator silicon.
- HBM or external memory.
- Power delivery circuitry.
- PCB design.
- Cooling system.
- PCIe or other host interface.
- Firmware.
- Operating-system drivers.
- AI runtime.
- Developer tools.
You then need to integrate everything into a production-ready board or server.
For an edge AI accelerator, the final product might instead be a compact module integrated into a camera, robot, industrial controller, or embedded computer.
15. Understand the economics of producing custom silicon
This is where the engineering project becomes a semiconductor business.
Custom AI silicon has substantial upfront costs.
Your expenses can include:
Expense What it covers Architecture System design and modeling Engineering RTL, verification and physical design EDA licensing Commercial chip-design software IP licensing Processor, interface and memory IP Prototyping FPGA platforms and validation Masks Manufacturing photomasks Wafer fabrication Manufacturing silicon Packaging Assembly and advanced packaging Testing Validation and manufacturing tests Software Drivers, compilers and runtime Production Boards, systems and supply chain Three possible business models
Model A: Fabless AI chip company
You design and own the accelerator and outsource fabrication.
This is the traditional approach used by many semiconductor companies.
Model B: Custom accelerator IP company
You develop reusable accelerator intellectual property and license it to other companies.
This can reduce the need to manufacture and sell complete physical products.
Model C: Vertical AI hardware company
You design the chip, manufacture the accelerator through external partners, build the software stack, and sell complete AI systems or cloud inference services.
This allows you to control more of the customer experience.
The importance of cost per inference
For a commercial AI accelerator, theoretical TOPS is not enough.
You need to understand the actual economics of running AI models.
Useful metrics include:
math\text{Energy per token}= \frac{\text{Energy consumed}}{\text{Tokens generated}}math\text{Cost per token}= \frac{\text{Total operating cost}}{\text{Tokens generated}}For an LLM inference chip, these can be more commercially meaningful than peak arithmetic throughput.
A chip that delivers fewer theoretical operations per second but produces tokens more efficiently could be a better commercial product.
16. How long does the entire process take?
The development schedule depends on architecture complexity, manufacturing technology, team experience, and whether you reuse existing IP.
A reasonable planning estimate for a first commercial ASIC is:
Illustrative development timeline
Example planning schedule, not a guaranteed industry benchmark.
Several stages overlap. Verification, software development and testing continue throughout the project.
A sophisticated high-performance accelerator can take considerably longer, especially if it requires advanced packaging, new memory technology, or multiple silicon revisions.
17. What tools and technologies would you need?
Here is a practical overview of the engineering ecosystem.
Category Examples Hardware description SystemVerilog, Verilog, Chisel Architecture modeling Python, C++, SystemC AI development PyTorch, ONNX RTL simulation Questa, VCS, Xcelium Logic synthesis Synopsys Design Compiler, Cadence Genus Physical design Cadence Innovus, Synopsys Fusion Compiler Formal verification JasperGold, VC Formal FPGA prototyping AMD Vivado, Intel Quartus Physical verification Siemens Calibre Chip layout GDSII, OASIS Software development C++, Python, LLVM Model compilation MLIR-based compiler infrastructure These are examples rather than a mandatory toolchain. Commercial ASIC development generally requires foundry-compatible EDA tools, licensed IP, and appropriate technology access.
For learning, three particularly useful resources are:
- CERN ASIC Support — Practical RTL-to-GDS implementation flows.
- Columbia ESP documentation — Open-source SoC and accelerator integration.
- AMD hardware development documentation — FPGA prototyping and accelerator development.
For additional background, AWS Semiconductor Design whitepaper describes the front-end and back-end chip-design process.
18. My recommendation: How I would approach building a specialized AI accelerator from scratch
I would not begin by trying to create a massive GPU competitor.
I would start with a narrowly defined AI workload and develop an accelerator that delivers a meaningful improvement over existing hardware.
Here is the approach I would take.
Suggested first-generation AI accelerator
(Practical starting concept)
Product concept: Specialized LLM inference accelerator
Target a specific category of transformer-based inference rather than trying to support every AI workload.
Initial architecture goals
- Matrix multiplication acceleration.
- INT8 and INT4 inference support.
- BF16 support where necessary.
- On-chip SRAM.
- External memory or HBM integration, depending on the product.
- Dedicated data-movement engines.
- Standard host connectivity.
- Programmable execution and scheduling.
Software goals
- PyTorch model integration.
- ONNX support.
- Model quantization.
- Custom inference runtime.
- Optimized attention and matrix multiplication kernels.
A staged development strategy
Phase 1 — Prove the concept
Build a Python performance simulator and benchmark actual AI models. Establish whether the proposed architecture has a meaningful advantage.
Phase 2 — Build a hardware prototype
Implement the important compute engines on an FPGA. Validate the hardware/software interface and test real model workloads.
Phase 3 — Develop the ASIC
Create RTL, verify the design, perform synthesis and physical implementation, and prepare for fabrication.
Phase 4 — Manufacture and validate
Work with a foundry and packaging provider to manufacture the chip, bring up the first silicon, and validate its performance.
Phase 5 — Commercialize
Build accelerator boards or modules, release the compiler and runtime, establish production, and sell to customers.
The critical principle is to validate the workload, architecture, and software before spending heavily on custom silicon.
19. The most important strategic question
There are several very different ways to approach this industry, and they require different levels of investment and expertise.
For example, designing a small edge AI accelerator is fundamentally different from developing a high-performance HBM-equipped data-center inference processor.
I'd suggest deciding which direction you want to pursue first:
1. What type of AI accelerator interests you most?
LLM and generative AI accelerators
Computer vision accelerators
Low-power edge AI accelerators
Robotics and autonomous systems accelerators
General-purpose AI accelerator
2. What is your primary objective?
Build a commercial semiconductor product/company
Develop a research prototype
Learn the complete chip-design process
Develop and license proprietary accelerator IP
3. How would you like to approach development?
Design everything from the ground up
Start with open-source hardware and modify it
Combine existing IP with custom silicon
Start with an FPGA prototype and transition to ASIC
[Develop my accelerator blueprint]
With those decisions, I can take this beyond the general semiconductor lifecycle and develop a concrete engineering blueprint, including the actual chip architecture, compute units, memory subsystem, software stack, manufacturing strategy, development team, and estimated investment required.
Pages cited in this answer13
- esp.cs.columbia.eduDocumentation — ESP — Open-Source SoC Platform
- semiconductors.orgDesign and front-end wafer fabrication are closely linked. During the design process, initial work is done on an electronic, software-based model of the chip. Fabs then provide test silicon to validate the electronic model and provide vital input on the design’s manufacturability. Close collaboration is critical: scaling up new processes involves inherent uncertainties in modeling and reaching target manufacturing yields. Design engineers and wafer fabrication engineers use PDKs—essentially documentation of fabrication processes—and work together to debug design features and verify that manufacturing processes and chip designs are compatible.
- downloads.regulations.govVia Regulatory Portal
- asicsupport.web.cern.chDigital implementation flows, from RTL to GDS - CERN ASIC Support and Foundry Services
- docs.aws.amazon.comSemiconductor Design on AWS - AWS Whitepaper
- docs.amd.comAMD Technical Information Portal
- vitisai.docs.amd.comVitis AI 6.3 - Release — Vitis AI Documentation
- synopsys.comWhat is Physical Design? – How it Works | Synopsys
- tsmc.comTaiwan Semiconductor Manufacturing Company Limited
- semiconductor.samsung.comFoundry Overview | Samsung Semiconductor Global
- gf.comGlobalFoundries | Semiconductor foundry and technology partner
- uciexpress.orgSpecifications | UCIe Consortium
- docs.amd.comAMD Technical Information Portal
Google AI ModeNov 1, 2026
Creating and producing a custom silicon accelerator—an Application-Specific Integrated Circuit (ASIC)—optimized for specialized AI workloads requires a multi-year, highly specialized engineering pipeline.
The end-to-end journey from a clean sheet to physical, production-ready AI chips is broken down into sequential phases below:
- Workload Analysis & Architectural Definition
- Target Identification: Analyze specific AI algorithms (e.g., LLM inference, diffusion models, sparse transformers) to pinpoint bottlenecks like memory bandwidth, matrix multiplication throughput (M A C cap M cap A cap C𝑀𝐴𝐶 operations), or specific quantization formats (F P 8 cap F cap P 8𝐹𝑃8, I N T 4 cap I cap N cap T 4𝐼𝑁𝑇4).
- Architectural Specification: Define the core compute topology, such as systolic arrays, tensor cores, on-chip SRAM hierarchy, and interconnect networks. Set strict targets for power envelope (Thermal Design Power, or T D P cap T cap D cap P𝑇𝐷𝑃), die area, and target performance metrics (TOPS or TFLOPs per watt).
- RTL Design & Microarchitecture Implementation
- Register-Transfer Level (RTL) Coding: Translate the architecture into hardware description languages (HDLs) like SystemVerilog or Chisel.
- IP Integration: Integrate specialized third-party or internal intellectual property (IP) blocks, including high-speed interface controllers like PCIe Gen6, CXL, and high-bandwidth memory controllers (HBM3e/HBM4).
- Functional Verification & Simulation
- Testbench Development: Build comprehensive verification environments using Universal Verification Methodology (UVM).
- Software-Hardware Co-Simulation: Emulate the RTL design on enterprise hardware emulation systems (such as Synopsys ZeBu or Cadence Palladium ) running actual AI workloads and software stacks before any silicon is cut.
- Logical Synthesis & Design for Test (DFT)
- Logic Synthesis: Use EDA tools (from vendors like Synopsys or Cadence ) to convert RTL code into a gate-level netlist mapped to a specific foundry's standard cell library.
- DFT Insertion: Embed test structures like boundary scan chains and Built-In Self-Test (BIST) engines to enable post-manufacturing fault screening.
- Physical Design & Timing Closure (RTL-to-GDSII)
- Floorplanning & Placement: Map out macro blocks, memory arrays, and IO pads on the physical die, managing power distribution networks (IR-drop analysis).
- Clock Tree Synthesis (CTS): Route clock signals uniformly across the die to minimize skew.
- Routing & Signoff: Connect all logic gates while satisfying strict Static Timing Analysis (STA), Design Rule Checking (DRC), and Layout Versus Schematic (LVS) verification. The final output is the GDSII/OASIS layout database.
- Tape-Out & Foundry Manufacturing
- Tape-Out: Deliver the final GDSII database to an advanced semiconductor foundry (such as TSMC or Intel Foundry).
- Wafer Fabrication: The foundry processes multi-patterning extreme ultraviolet (EUV) lithography across months, building dozens of metal layers on silicon wafers.
- Advanced Packaging & Chiplet Integration
- 2.5D/3D Packaging: For high-performance AI accelerators, mount the compute die alongside HBM stacks onto a silicon interposer using advanced packaging technologies (like TSMC CoWoS).
- Post-Silicon Validation & Bring-Up
- Laboratory Bring-Up: Power on the first physical chips on custom evaluation boards using logic analyzers and oscilloscopes.
- Software Stack Tuning: Calibrate firmware, low-level drivers, and runtime compilers (integrating with PyTorch/TensorFlow backends) to achieve functional stability and peak performance.
If you want to dig deeper, let me know if you would like to explore:
The economics and NRE (Non-Recurring Engineering) costs associated with advanced process nodes (e.g., 3nm/2nm)Specific architectural choices for memory vs. compute-bound AI models
Pages cited in this answer1
Also asked as
- I need to understand the complete lifecycle of developing and manufacturing bespoke AI chips tailored to unique workloads. What does that workflow look like?