Originally published on X in May 2026. Updated and archived here as part of Silicon Valley Trivopaedia.

For years, leading AI companies focused primarily on software optimization. NVIDIA’s CUDA ecosystem, along with mature libraries like cuDNN, TensorRT, and NCCL, was so dominant that most organizations believed they could maintain a competitive edge through software alone. “Let NVIDIA handle the hardware; we’ll optimize on top of it” was the prevailing mindset.

That assumption began to break down around 2023–2024.

The explosion in model scale dramatically increased both training and inference costs. More importantly, inference — not training — became the dominant cost driver in real-world production systems. Software optimizations helped, but they eventually hit hard limits in memory bandwidth, data-movement overhead, and low-precision compute efficiency. These bottlenecks could no longer be fully solved at the software layer.

At the same time, NVIDIA’s supply constraints and premium pricing during the H100/H200 era created serious operational and financial pressure. Hyperscalers investing tens of billions of dollars in AI infrastructure started treating GPU cost and power consumption as core strategic risks, not just procurement issues.

Equally important was workload specialization. Meta’s recommendation systems, Google’s search and Gemini workloads, Amazon’s Bedrock services, and Tesla’s FSD training have very different computational characteristics than generic large language models. These workloads benefit from hardware tailored to specific patterns — massive KV-cache handling, sparse operations, aggressive low-precision formats — which general-purpose GPUs are not optimally designed for.

Advances in chiplet architecture, advanced packaging, and the maturity of TSMC’s 3nm process finally made it technically and economically viable for these companies to design their own silicon. The potential gains in power efficiency, silicon cost, and performance per dollar (often estimated at 30–50%+) became too significant to ignore.

As a result, several hyperscalers moved from “software on top of NVIDIA” to building custom ASICs purpose-built for their specific needs. Below is the state of the major efforts, originally written in mid-2026 and lightly updated after Hot Chips 2026.

1. Meta — MTIA Series: The Most Aggressive Multi-Generation Push

In March 2026, Meta announced four generations of its Meta Training and Inference Accelerator — MTIA 300, 400, 450, and 500 — developed with Broadcom on a chiplet-based approach.

• MTIA 300 is in production, optimized for ranking and recommendation training.

• MTIA 400 expands coverage to general GenAI workloads and supports large-scale 72-chip configurations. As of Hot Chips 2026 (late August), Meta presented MTIA 400 as the near-term workhorse, with rollout in 2026.

• MTIA 450 and 500 are heavily optimized for GenAI inference. The standout improvement is memory bandwidth: MTIA 450 targets 18.4 TB/s, while MTIA 500 targets 27.6 TB/s with up to 512 GB of HBM. That directly helps large-context inference and high batch-size serving by keeping more KV cache in fast memory.

Meta’s ability to stack four generations in a short window reflects both the urgency of GenAI inference demand and the advantages of its chiplet + Broadcom partnership. The chips are already running at scale for Facebook and Instagram recommendations as well as Llama inference, with plans to expand further into training. Mass deployment of 450/500 is still slated for 2027.

2. Microsoft — Maia 200: Focused on Inference Economics

Announced in January 2026, Maia 200 is Microsoft’s clearest statement yet on custom silicon. It is purpose-built for inference, specifically the economics of token generation.

Built on TSMC 3nm with native FP8/FP4 tensor cores, it features 216 GB of HBM3e at 7 TB/s plus a large on-chip SRAM pool. Microsoft has claimed meaningful advantages versus competing custom chips from Amazon and Google on certain inference workloads.

By August 2026, the story had moved from announcement to architecture disclosure and production use:

• Microsoft presented Maia 200 at Hot Chips 2026 (750W TDP class, ~10 PFLOPS FP4 in disclosed figures).

• An arXiv paper described it as a software-defined dataflow system.

• Microsoft’s own MAI models are being trained and served with co-design on Maia 200, and some Copilot traffic is already routing to first-party models running on this stack.

The chip is no longer a slide. It is inside Azure.

3. Amazon — Trainium3: The Most Production-Ready at Scale — and Still Buying NVIDIA

Trainium3, unveiled at re:Invent 2025 and now in broad production, remains one of the most mature custom AI chips in real-world use. Built on TSMC 3nm, it powers the Trn3 UltraServer, which can integrate up to 144 chips in a single system.

It delivers up to 4.4× higher performance and roughly 4× better performance per watt versus Trainium2, with strong support for MoE architectures and low-precision formats (MXFP8/MXFP4). Multiple Bedrock customers have reported cost reductions of up to 50% versus GPU-based alternatives. Amazon’s advantage is not just the die — it is the tightly integrated server + software system.

The August 2026 update matters: Amazon is not walking away from NVIDIA. AWS committed to another 2 million NVIDIA GPUs for 2027–2028 (Blackwell Ultra / Rubin / Rubin Ultra), on top of an earlier ~1 million commitment. Custom silicon is a cost-and-control weapon. It is not a full substitution strategy — at least not yet.

4. Google — Ironwood, Then TPU v8

Google’s seventh-generation TPU, Ironwood, was designed for the “age of inference” while keeping strong training capability. It claimed up to 10× peak performance over TPU v5p and more than 4× per-chip improvement versus Trillium (v6e). Its distinctive strength is extreme scale: Superpods connecting thousands of chips, backed by Pathways.

By spring/summer 2026, Google had already moved to an eighth-generation split:

• TPU 8t for training

• TPU 8i for inference

Hot Chips 2026 made the systems-level ambition explicit: a ~9,600-chip training domain, dedicated inference silicon to avoid dark silicon, and continued co-design of networking + orchestration. Google’s bet is not “a faster chip.” It is treating tens or hundreds of thousands of chips as one coherent computer.

Ironwood is the previous chapter. TPU v8 is the current one.

5. Other Notable Efforts

Tesla Dojo. Continues to evolve for FSD training and large-scale internal AI workloads. Development on later Dojo generations resumed in 2026 after Tesla stabilized its AI5 path.

Apple Baltra. Working with Broadcom on a server-grade inference chip (2026–2027 window), chiplet design on TSMC 3nm, aimed at Apple Intelligence workloads.

OpenAI. The first custom chip, previously discussed as Project Titan, surfaced publicly as Jalapeño (with Broadcom). Engineering samples were running in 2026; Hot Chips 2026 and OpenAI’s own disclosures claimed better throughput per kilowatt and lower token latency than NVIDIA GB200/GB300-class systems on selected public inference benchmarks. Initial deployment is targeted for later 2026, with next generations already in design. This is the most important update since the original May post.

Broadcom. Still the critical enabler — design and supply partner for Meta, OpenAI, and others. Custom AI accelerator revenue has become a core growth engine, not a side business.

6. Key Takeaways

The shift toward custom ASICs is not merely about raw performance. It reflects a recognition that software optimization alone has hit diminishing returns against fundamental hardware constraints—especially memory bandwidth, power efficiency, and workload-specific compute patterns.

Common technical themes:

• Chiplet architectures

• Aggressive HBM (HBM3e now, HBM4 incoming)

• Native low-precision formats (FP8, FP4, MXFP)

• TSMC 3nm as the common manufacturing substrate

• Differentiation at architecture and systems level (rack, fabric, compiler, serving stack)

For NVIDIA, this is a long-term structural challenge even while CUDA, CUDA-X, and broad applicability remain powerful moats. Amazon’s latest GPU order is the clearest proof that NVIDIA is not being replaced overnight. It is being complemented and selectively displaced where workloads are stable enough to justify a custom die.

For hyperscalers, custom silicon has moved from an interesting side project to a core strategic capability: control over cost, performance, power, and supply-chain risk.

The next 18–24 months will likely bring even more aggressive roadmaps and greater heterogeneity in AI infrastructure. Each major player is doubling down on hardware tailored to its own strengths. The industry is no longer one GPU architecture with many software stacks. It is becoming many architectures with competing full stacks.

References

• Meta. (March 11, 2026). Expanding Meta’s Custom Silicon to Power Our AI Workloads.

• ServeTheHome. (August 25, 2026). Meta’s MTIA Custom AI Silicon at Hot Chips 2026.

• Microsoft. (January 26, 2026). Maia 200: The AI accelerator built for inference.

• ServeTheHome. (August 25, 2026). Microsoft’s Maia 200 AI Accelerator at Hot Chips 2026.

• Hoefler et al. (August 2026). Maia 200: A Software Defined Dataflow System for Large-scale AI Acceleration. arXiv:2608.24664.

• AWS. (December 2025). Amazon EC2 Trn3 UltraServers. re:Invent 2025.

• TechCrunch / NVIDIA earnings week. (August 26, 2026). Amazon’s expanded NVIDIA GPU commitment.

• Google Cloud. Ironwood TPU materials (2025–2026) and TPU 8t / 8i technical deep dive (April 2026).

• ServeTheHome. (August 25, 2026). Google’s TPUv8s for Training and Inference at Hot Chips 2026.

• OpenAI. (June 24, 2026). OpenAI and Broadcom unveil LLM-optimized inference chip.

• OpenAI. (August 24–25, 2026). Jalapeño performance disclosures / Hot Chips coverage.

• Reuters / HPCwire. (March 11, 2026). Meta in-house AI chip coverage.

A shorter version of this essay first appeared as an [X Article](https://x.com/i/article/2092506475327303680) on August 26, 2026.

Only a few years ago, Nvidia’s GTC was mostly a conference for developers and researchers. Now ordinary investors and the broader public put it on their calendars. “GPU” followed the same path. What once belonged to gamers and specialists is now everyday language whenever people talk about AI.

Something similar is happening to Hot Chips. As recently as last year, it was still the kind of technical conference you followed mainly if you cared about the silicon itself—architecture and circuit talks, RISC-V, and, for people in the interconnect world, optics and co-packaged optics (CPO). A colleague who works on optics told me they went last year for CPO, not for the consumer-AI circus. This year the same conference suddenly draws not only engineers but also tech media and influencers. If GTC has become the public stage for AI chips, Hot Chips has become the design stage, now open to wider markets and audiences. That is part of why announcements like OpenAI’s Jalapeño benchmarks and Cerebras’s wafer-scale roadmap drew attention in the same week.

The underlying agenda did not appear overnight. Optics and CPO were already on the program because racks are running into power, reach, and bandwidth walls that a faster GPU alone does not solve. What changed is who now treats those talks as market news.

Saying that AI semiconductors are “hot” already feels dated. The more accurate statement is that AI chips have become industrial infrastructure. That infrastructure is not converging on a single chip, a single company, or a single architecture. If anything, the opposite is true. In the AI world, Nvidia GPUs are driving; complementary technologies that GPUs alone cannot fully explain are growing at the same time.

1. Why Nvidia Is Number One—and Why It Is Not Number One Everywhere

Nvidia’s lead is not accidental. GPUs are strong at massive parallel compute, proven on training workloads, and backed by a deep software ecosystem centered on CUDA. The silicon is not the only factor. Productivity across building and deploying models on top of it is high. So whenever the question arises—“Can this replace Nvidia?”—people instinctively use the GPU as the reference point.

A GPU is not optimal for every domain. Training and inference are different jobs. Large-scale training prioritizes huge matrix math and flexible programming. Inference prioritizes power, latency, cost per token, and how stably you can serve traffic. In agentic systems, where models are called repeatedly, that difference compounds. One slow response is rarely just one slow response.

That is why complementary approaches have been studied for years: heterogeneous CPU–GPU systems, NPUs tuned for low-power inference, LPUs specialized for language-model serving, TPUs optimized for tensor operations, and ASICs carved for a specific service path. The names differ; the direction does not. The goal is not to deny what GPUs do well. It is to cover the places where GPUs are relatively expensive or slow.

2. The Trap in “Is It Better Than Nvidia?”

Whenever a company announces new silicon, the first question is almost automatic: Is it better than Nvidia’s GPU? Can it replace Nvidia?

The real answer is both yes and no. On specific workloads, specialized chips clearly win—performance per watt, ultra-low latency, narrow operator patterns, deterministic execution. On generality, developer ecosystem, model coverage, large-scale training, and full-stack maturity, Nvidia’s moat is still thick.

The problem is that the question is too binary. Markets and media remain comfortable with a single-winner frame. Real systems are not. A strong specialized chip is often not a device that erases Nvidia. It is a device that fills a gap in the AI world Nvidia is driving. The competitive key is shifting from “who builds the stronger single chip” to “who binds different chips into one coherent system.”

That fusion is concrete:

• Which parts of a model run on GPU, and which on NPU, LPU, TPU, or ASIC

• Compilers and runtimes that place work without forcing every developer to micromanage hardware

• Memory and interconnects that move data as little—and as fast—as possible between chips

• Total cost of ownership that includes power, cooling, floor space, operations, and development speed, not just chip price

When those links break, the advantage of specialized silicon disappears quickly.

3. Cerebras: Not a Replacement—an Architecture That Fills the Speed Gap

In my complementary view, one of the cases I have emphasized most consistently—and for the longest time—is Cerebras. The core point I have repeated since before the IPO is simple:

Cerebras is not a GPU replacement. It is a complementary solution meant to work with GPUs.

GPUs excel at throughput but lag on per-user token speed. Cerebras fills that speed gap. Together, they can raise both speed and throughput. The point is not who beats whom. It is how the roles are split.

That claim comes from architecture. Cerebras’s essence is extreme data locality. A full 300 mm wafer is treated as essentially one giant chip, with hundreds of thousands to roughly 900,000 small cores and SRAM placed next to them. Model weights can stay resident in on-chip SRAM, sharply reducing the memory-wall tax and inter-chip communication overhead GPUs repeatedly pay. On WSE-3, aggregate on-chip SRAM bandwidth is described around 21 PB/s; on WSE-3 Turbo, roughly on the order of double that. That is a different order of magnitude from typical GPU–HBM paths.

Experienced chip designers often have the same first reaction: yield will be catastrophic; you cannot test something that big or survive that many defects. Then you look at the actual design—tiny identical cores, massive redundancy, a dynamic re-routing fabric that maps around defective regions. It is not SRAM dumped on a wafer. It is cores and memory treated as one unit, with system-level yield engineered in. What first looked insane starts to look structurally coherent.

The recent CS-4 pushes that philosophy to rack scale: wafer-scale engines on the Nexus platform, aimed at ultra-fast inference and hyperscale deployment. Vendor claims emphasize large gains in token generation speed and tokens per watt versus prior generations. More important is the design intent: disaggregated inference, with prefill on GPUs or other accelerators and decode on Cerebras. The collaboration with AMD Helios is a concrete example of that split.

The CS-6 roadmap shown at Hot Chips 2026 goes one step further: keep wafer-scale SRAM and compute-fabric locality, then add 3D-stacked DRAM through ultra-high-bandwidth vertical connections to break the capacity ceiling. SRAM is fast but area-hungry; capacity has always been the trade-off. CS-6 is an attempt to gain capacity without throwing away the locality that makes the architecture special. Whether it works at production economics remains to be proven. The direction, however, is consistent.

Cerebras’s own wording at Hot Chips is the key sentence: by integrating wafer-scale SRAM and compute with 3D-stacked DRAM through ultra-high-bandwidth connections, CS-6 is designed to expand memory capacity without sacrificing the locality that makes wafer-scale fast.

OpenAI’s moves fit the same frame. Jalapeño is a custom inference ASIC for scaling internal infrastructure and improving efficiency. Paths like /ultrafast lean on Cerebras’s hardware characteristics for ultra-low latency. Different roles, operated in parallel—not one killing the other, but segments inside a large inference market.

What Jalapeño Shows

On August 25, 2026, OpenAI published the first public benchmarks for Jalapeño, the inference chip co-developed with Broadcom, at Hot Chips.

On SemiAnalysis’s InferenceX suite, OpenAI reported roughly 1.5–1.9× more AI work per watt and 1.7–3.6× lower end-to-end latency versus Nvidia GB200/GB300 rack systems across models such as GPT-OSS 120B, DeepSeek R1, and Kimi K2.5. Jalapeño is a ~700W-class part, inference-only, with limited deployment targeted later this year and broader rollout in 2027. [TechCrunch]

Reading those numbers as the end of the Nvidia era is premature. Results shift with package TDP versus facility power. Nvidia’s next generation was not in the comparison. Jalapeño does not replace training. Still, the message is clear: when a model company designs silicon around its own inference path, it can attack gaps left by general-purpose GPU design.

The same day’s context also reconfirms that OpenAI is not putting all inference on one class of hardware. One path optimizes for scale and cost; another for extreme speed. While specialized-chip announcements are often consumed as winner-replacement stories, the field is already practicing role division.

Specialization Is Connection, Not Isolation

This is not only an OpenAI–Cerebras story. Google splits TPUs across training and inference and diversifies design partners beyond a single supplier. AWS Inferentia/Trainium, Microsoft Maia, and Meta MTIA follow the same logic: bake repeated service patterns into silicon and cut power and cost.

Nvidia is not blind to the shift. It has moved from selling GPUs alone toward CPU–GPU combinations, rack-scale systems, and stronger inference software. The CUDA lock-in remains powerful. But the meaning of that moat is changing. The fight is less “GPU versus anti-GPU” and more “how GPU-centered systems coexist with specialized accelerators.”

An Investment Stance for a Growing Pie

Market-share figures vary by research house. What does not vary much is that Nvidia remains the dominant data-center AI accelerator franchise, while custom ASICs and specialized inference are growing quickly. The substance is simple: this is not a market where Nvidia must die for others to live. Aggregate demand is rising fast enough for multiple designs to find room at once.

The AI-semiconductor investment direction I take is this: for now, the market looks complementary rather than winner-take-all, so I keep a larger core position in Nvidia and add holdings in the companies that cover the layers GPUs do not fully own. That is different from spraying capital across many names on the theory that someone will win eventually.

What “complementary” means in practice:

• Nvidia still leads in training, general-purpose stacks, and the software ecosystem.

• AMD is the other merchant-GPU and rack-scale path.

• Broadcom is the custom-ASIC design and implementation layer behind chips such as OpenAI’s Jalapeño and Google’s TPUs.

• Google runs a separate deployment path through its own TPUs and cloud.

• Cerebras covers the ultra-low-latency inference gap with wafer-scale architecture.

These companies take demand in different parts of the system more than they erase one another.

In the portfolio, Nvidia’s weight is still larger than the sum of the other related holdings. The judgment that Nvidia is the core axis has not changed. At the same time, exposure to the other companies that can lead different AI-chip axes—AMD, Broadcom, Google, Cerebras—is being increased as the technology and market expand. Relative to Cerebras, Nvidia has been on the order of about ten times; AMD and Google roughly similar to Cerebras; Broadcom closer to AMD plus Cerebras combined.

The exact numbers matter less than the allocation sense: do not chase a single flag, and do not lose the center.

This is not “anyone can win” diversification. It is a stance aligned with a pie that is growing complementarily for now. One company may announce a new chip today; another may claim supremacy a few days later. Inside that noise, training remains largely GPU-centered, inference is differentiating quickly, and software plus system integration either completes or offsets hardware advantages.

Whether to buy any specific stock is ultimately a personal judgment. The weights and direction above are **not a recommendation**—only one example of translating a “not a replacement war, but complementary pie expansion” view into capital allocation. In this phase, the better question is not “Does this chip kill Nvidia?” but “Which segment of the full system does this chip make cheaper and faster?” When those segments grow together for a while, keeping the core axis while scaling the others is a coherent response.

What to Watch Next

The next months and years will be an era of chip announcements and, at the same time, an era of orchestration. What matters more than who claims larger FLOPS is who splits prefill and decode cleanly, who moves KV cache less, and who actually runs services inside the physical limits of power and cooling. That GTC and Hot Chips have both grown is itself a signal that these competitive and design debates have moved beyond a narrow conference wall.

GPUs are already common knowledge. The next piece of common knowledge may be this: the end state of the AI semiconductor contest is less likely to be one company’s flag than many chips complementing one another inside a rack, a cloud, and a service path—GPUs holding the center of throughput, wafer-scale designs like Cerebras filling the speed gap, and custom ASICs such as Jalapeño and TPUs bending the cost curve for specific services.

The side that designs and operates that fusion best will hold the more durable advantage—longer than any single chip’s scoreboard. Translated into investment terms: for now, keep the core on that complementary growth, and keep building the satellite axes around it.

Subscribe now

I started SVTrivo with a simple premise: the most valuable insights in Silicon Valley rarely fit into a single tweet or a press release.

Instead of rewashing daily tech news, this publication serves as a permanent research vault. We focus purely on how technology, capital, and real-world execution clash to shape the next market cycles.

What You Actually Get

Forget surface-level commentary. Here is what shows up in your inbox:

• Tech & AI Infrastructure. We look past the immediate hype. That means tracking hyperscaler CAPEX, physical supply chain bottlenecks, and actual enterprise deployment.

• Capital & Valuation. Breaking down VC deal mechanics, private liquidity pressures, and assets that public markets are completely mispricing.

• Execution Playbooks. Real, tactical teardowns on how elite founders scale when the macro backdrop turns messy.

Membership Options

Free

• 1 or 2 long-form essays every month on major structural shifts across tech and finance.

Paid Member ($15/month or $150/year)

• Weekly Deep Dives: Rigorous, data-backed teardowns on key sectors and high-growth companies.

• Real-Time Commentary: Raw, immediate analysis published right after major earnings, policy shifts, or unexpected market moves.

• Full Archive Access: Complete access to every past model, framework, and deep dive.

• Subscriber Threads: Direct access to private Q&A threads and subscriber-only discussions.

About the Author

I’ve spent years tracking global tech dynamics—building frameworks alongside a community of over 14,000 founders, investors, and operators on X. SVTrivo is where those ongoing conversations turn into actionable, high-conviction research.

Filtering out the noise. Compounding the signal.

Subscribe today to get the next report delivered straight to your inbox.