At a recent AI meetup, someone asked:
"With models getting more capable and cheaper every month, where should startup founders focus?"
It's a fair question. The short answer is: yes---but not where most people are looking.
Raw AI capability is becoming abundant. Ownership of atoms, data, workflows, and distribution is not. That's where the value accrues. That's what we call a moat. And without a moat, building a durable business is hard---if not impossible.
What follows is what I took away from the meetup, combined with my own analysis of what moats in the AI stack actually look like, viewed through a financial and investment lens.
1. Physical infrastructure: the scarcity layer
Every AI system ultimately runs on fundamentals: compute, networking, land, cooling, and power. This "atoms" layer is the most capital-intensive part of the stack and, historically, one of the most defensible.
Moats at this layer come from a few sources:
- Capital intensity: only a handful of players can spend tens or hundreds of billions annually on data centers, chips, and grid connections.
- Supply constraints: access to advanced GPUs, high-bandwidth interconnects, and reliable power is structurally scarce.
- Operational excellence: keeping massive compute clusters highly utilized, cool, and online is genuinely hard.
- Geography and regulation: permits, data sovereignty, and grid access slow down would-be competitors as they attempt to scale.
As long as AI compute demand grows faster than supply, this scarcity layer will retain pricing power and generate attractive returns.
GPUs, clouds, and power
NVIDIA's GPUs have become the default compute substrate for both training and inference---not just because of silicon performance, but because of CUDA, a rich software ecosystem, and tight integration across hardware and networking. Replacing CUDA with an alternative standard is expensive and time-consuming. AMD and custom silicon from Google and Amazon matter, but so far they reinforce a winner-take-most dynamic rather than true commoditization.
On top of this, hyperscalers---AWS, Azure, and Google Cloud---bundle compute, storage, networking, and compliance into global AI platforms. Specialized GPU clouds like CoreWeave fill the gaps, offering faster access to the latest GPUs and more tailored economics where hyperscalers cannot move quickly enough.
A new bottleneck, however, is emerging: power. Data centers are now competing for grid capacity, long-term power contracts, and on-site generation. Quietly, this creates new moats for whoever can reliably feed these power-hungry GPU clusters.
2. Software infrastructure: foundation models without a monopoly?
If physical infrastructure is about scarcity, software infrastructure is about capability and leverage. This layer includes foundation models, hosting, orchestration, MLOps/LLMOps, and developer tooling.
Leaders include OpenAI, Anthropic, Google Gemini, and Meta's LLaMA ecosystem. But unlike classic infrastructure monopolies, several forces push against a clean winner-takes-all outcome:
- Published AI research diffuses quickly.
- Open-source and open-weight models are rapidly closing the gap.
- Clever algorithmic approaches (deep seek) can outperform brute-force scaling.
- Enterprises are increasingly going multi-model for cost, redundancy, and sovereignty.
- Regulators tend to prefer diversity over concentration.
A winner-take-most outcome is still possible---particularly if one provider maintains durable cost and capability leadership, or if default models are hard-bundled into operating systems, productivity suites, search, or devices. However, as the LLM technology evolves, the difference between these foundation models is becoming increasingly minor. Its not uncommon to see LLM users attribute stylistic preferences of one LLM over the other.
Around the models, a tooling ecosystem is maturing: vector databases (Pinecone, Weaviate), observability and LLMOps (LangSmith), agent frameworks and protocols (LangChain, MCP). These tools remove friction for developers, but switching costs are often modest, and popular features tend to be absorbed into open-source alternatives over time.
In this layer, moats come less from individual features and more from ecosystem gravity and integration depth.
3. LLM extenders: vertical AI and "wrapper risk"
This is where most startups are currently playing. LLM extenders take foundation models and apply them to specific verticals or workflows---law, support, healthcare documentation, financial analysis, insurance, and more.
Examples include Harvey (legal), Glean (enterprise knowledge), Ada (support automation), and Abridge (clinical documentation). These businesses can be defensible, but only when they truly own:
- Workflow: they sit directly in the critical path of daily operations.
- Proprietary assets: contracts, annotations, feedback loops, and integrations others cannot replicate.
- Outcomes: guarantees around accuracy, compliance, or liability.
- Trust and regulation: approvals and brand in high-stakes domains.
- Systems of record: deep hooks into EHRs, ERPs, claims, billing, or CRMs.
In this world, the LLM is an ingredient---not the product.
The real risk for startups is being "just a wrapper": a polished interface on top of a general-purpose model, with no unique data, no workflow ownership, and no hard-to-switch integrations.
As base models become cheaper and better, wrapper startups face three pressures:
- Foundation model providers ship similar features natively.
- Enterprises build internal versions using APIs.
- Incumbents bundle equivalent capabilities into products customers already pay for.
A simple gut check: if you rip the LLM out of your product and little proprietary value remains, your moat is thin.
4. LLM adapters: incumbents playing defense and offense
Then there are incumbents that already own distribution and workflows and are now infusing them with AI---Microsoft Copilot in Office, Salesforce Einstein in CRM, Adobe Firefly in design, GitHub Copilot for developers.
In the short term, these players face real challenges: high inference costs, customers expecting AI to be "free," and competitive pressure that makes price increases difficult. Margins compress. Founders often dismiss them with, "What they ship sucks."
Over time, however, incumbents can:
- Bundle AI into existing products.
- Increase ARPU via premium tiers.
- Reduce churn by making core products meaningfully better.
- Optimize costs through routing, fine-tuning, and scale.
Because they already own the customer, the data, and the workflow, incumbents are often better positioned than startups to capture AI value---even if they move slowly and their first iterations are mediocre.
AI-native challengers still have a path, but usually only when they reimagine the workflow entirely, not when they add features incumbents can copy and bundle.
5. A simple moat test for AI startups
Across the AI stack, a few truths are becoming clear:
- AI capability itself is not scarce.
- Moats are not stationary; as models improve, moats move and the value chain shifts.
- Durability comes from ownership, not feature novelty.
A VC suggested a simple stress test: If base models become dramatically cheaper and more capable next year, what happens to your company?
If the honest answer is, "Customers can't leave without real operational pain, regulatory risk, or migration cost," there may be a moat.
If the answer is, "We'll ship more features," the moat is likely very thin.
6. Which company has the best AI moat today?
Competition is fierce at every layer. Every software company is now rebranded as an AI company, and it's increasingly difficult to separate signal from noise. That said, there is a meaningful structural difference between Google and most other players in the AI value chain.
Google is unusual because it owns every critical layer of the AI stack, and it built them long before AI was fashionable.
Silicon: Google built before it was cool
Google introduced TPUs in 2016---not because GPUs were scarce, but because they were suboptimal for Google's workloads: search ranking, ads, and translation.
TPUs were internal-only for years. Google didn't monetize them until much later. That's telling. This wasn't a "we need AI exposure" move; it was an operational necessity.
Contrast that with companies scrambling for NVIDIA allocation in 2023--2024.
Even today, many Google workloads do not depend on NVIDIA at all, insulating the company from GPU pricing cycles. TPUs are tightly co-designed with TensorFlow, compiler stacks, networking, and Google's data-center topology.
Power and data centers: AI as a first-class workload
Google began optimizing power, cooling, and placement for machine learning over a decade ago. DeepMind-driven optimizations reportedly reduced cooling energy usage by roughly 40%.
This is recursive advantage: AI improving the infrastructure that runs AI.
Compare that with companies only now discovering that power---not GPUs---is the binding constraint, and hyperscalers signing rushed nuclear or gas contracts under pressure. Google didn't panic-buy power. They designed for it.
Models: inventing the stack, then giving it away
Google's AI paradox is that it invented much of the modern stack, published it, and then appeared to have gotten "outgunned" commercially.
Transformers, BERT, T5, PaLM---all originated internally. Even today, GPT-style models rest heavily on Google-published research.
The critique that "Google missed the ChatGPT moment" misses the structural point. Google optimized for defensive integration across search, ads, and Android. Moreover, Google probably didn't want to cannibalize their search engine franchise.
In contrast, OpenAI optimized for distribution via novelty. Those are very different games.
Distribution: AI as a default, not a product
Google's biggest moat is boring and ubiquitous: Search, Gmail, Docs, Android, Chrome, YouTube.
For Google to be successful, Gemini doesn't need to be the best model. It needs to be good enough, cheap enough, and ubiquitous.
That's classic bundling power. AI becomes table stakes, and marginal inference cost is amortized across massive ad and subscription revenue. One could argue that Google is the only player whose marginal cost of inference approaches zero. If AI does become table stakes, Google could probably win as a low cost supplier of inference on LLMs given its dominance in the value chain.
Buying into AI versus building it
Other players have taken a different route: spending cash, raising debt, or pulling in massive private capital to buy relevance.
Microsoft is the strongest example. Its partnership with OpenAI was a brilliant distribution move---but it's still not vertical integration.
Microsoft didn't build GPT. It wrote a ~$13B check and secured exclusive cloud hosting. Microsoft still depends heavily on NVIDIA GPUs, doesn't fully control OpenAI's roadmap, and faces margin pressure because inference costs don't vanish at scale. This is partnership leverage, not ownership.
Meta offers another contrast. Despite heavy investment, it failed to dominate AI infrastructure or models. Instead, it open-sourced LLaMA---effectively choosing ecosystem gravity over cloud economics. That's a smart defensive move, but it's still vertical only at the application layer.
Legacy players like IBM and Oracle have struggled even more. They announced partnerships, rebranded products, and promised AI transformations---but lacked the research capability, infrastructure depth, and tooling ecosystems to matter. The shopping-spree playbook they are using is familiar:
- Acquire AI startups
- Rebrand existing products
- Sell "AI-powered" features without changing workflows
The outcome is predictable: brief novelty, incremental revenue, shrinking margins, bundled alternatives elsewhere, and disappearing pricing power. It's wrapper risk at enterprise scale!
Why Google is structurally different
Compared to everyone else, Google stands apart:
- No acquisitions to buy relevance
- No loading up on debt to buy its place in the value chain
- No dependency on third-party models
- No need for circular financing to get news headlines
- No reliance on short-term monetization optics
- Willingness to wait years for compounding advantage from their AI investments
It looked slow---until it wasn't!