Where Are the Moats in AI?

At a recent AI meetup, someone asked:

"With models getting more capable and cheaper every month, where should startup founders focus?"

It's a fair question. The short answer is: yes---but not where most people are looking.

Raw AI capability is becoming abundant. Ownership of atoms, data, workflows, and distribution is not. That's where the value accrues. That's what we call a moat. And without a moat, building a durable business is hard---if not impossible.

What follows is what I took away from the meetup, combined with my own analysis of what moats in the AI stack actually look like, viewed through a financial and investment lens.

1. Physical infrastructure: the scarcity layer

Every AI system ultimately runs on fundamentals: compute, networking, land, cooling, and power. This "atoms" layer is the most capital-intensive part of the stack and, historically, one of the most defensible.

Moats at this layer come from a few sources:

• Capital intensity: only a handful of players can spend tens or hundreds of billions annually on data centers, chips, and grid connections.
• Supply constraints: access to advanced GPUs, high-bandwidth interconnects, and reliable power is structurally scarce.
• Operational excellence: keeping massive compute clusters highly utilized, cool, and online is genuinely hard.
• Geography and regulation: permits, data sovereignty, and grid access slow down would-be competitors as they attempt to scale.

As long as AI compute demand grows faster than supply, this scarcity layer is likely to retain pricing power. That said, these dynamics can reverse quickly: supply can catch up, demand can disappoint, and today's scarcity premium can compress. None of this is a forecast of returns for any company.

GPUs, clouds, and power

NVIDIA's GPUs have become the default compute substrate for both training and inference---not just because of silicon performance, but because of CUDA, a rich software ecosystem, and tight integration across hardware and networking. Replacing CUDA with an alternative standard is expensive and time-consuming. AMD and custom silicon from Google and Amazon matter, but so far they reinforce a winner-take-most dynamic rather than true commoditization.

On top of this, hyperscalers---AWS, Azure, and Google Cloud---bundle compute, storage, networking, and compliance into global AI platforms. Specialized GPU clouds like CoreWeave fill the gaps, offering faster access to the latest GPUs and more tailored economics where hyperscalers cannot move quickly enough.

A new bottleneck, however, is emerging: power. Data centers are now competing for grid capacity, long-term power contracts, and on-site generation. Quietly, this creates new moats for whoever can reliably feed these power-hungry GPU clusters.

2. Software infrastructure: foundation models without a monopoly?

If physical infrastructure is about scarcity, software infrastructure is about capability and leverage. This layer includes foundation models, hosting, orchestration, MLOps/LLMOps, and developer tooling.

Leaders include OpenAI, Anthropic, Google Gemini, and Meta's LLaMA ecosystem. But unlike classic infrastructure monopolies, several forces push against a clean winner-takes-all outcome:

• Published AI research diffuses quickly.
• Open-source and open-weight models are rapidly closing the gap.
• Clever algorithmic approaches (deep seek) can outperform brute-force scaling.
• Enterprises are increasingly going multi-model for cost, redundancy, and sovereignty.
• Regulators tend to prefer diversity over concentration.

A winner-take-most outcome is still possible---particularly if one provider maintains durable cost and capability leadership, or if default models are hard-bundled into operating systems, productivity suites, search, or devices. However, as the LLM technology evolves, the difference between these foundation models is becoming increasingly minor. Its not uncommon to see LLM users attribute stylistic preferences of one LLM over the other.

Around the models, a tooling ecosystem is maturing: vector databases (Pinecone, Weaviate), observability and LLMOps (LangSmith), agent frameworks and protocols (LangChain, MCP). These tools remove friction for developers, but switching costs are often modest, and popular features tend to be absorbed into open-source alternatives over time.

In this layer, moats come less from individual features and more from ecosystem gravity and integration depth.

3. LLM extenders: vertical AI and "wrapper risk"

This is where most startups are currently playing. LLM extenders take foundation models and apply them to specific verticals or workflows---law, support, healthcare documentation, financial analysis, insurance, and more.

Examples include Harvey (legal), Glean (enterprise knowledge), Ada (support automation), and Abridge (clinical documentation). These businesses can be defensible, but only when they truly own:

• Workflow: they sit directly in the critical path of daily operations.
• Proprietary assets: contracts, annotations, feedback loops, and integrations others cannot replicate.
• Outcomes: guarantees around accuracy, compliance, or liability.
• Trust and regulation: approvals and brand in high-stakes domains.
• Systems of record: deep hooks into EHRs, ERPs, claims, billing, or CRMs.

In this world, the LLM is an ingredient---not the product.

The real risk for startups is being "just a wrapper": a polished interface on top of a general-purpose model, with no unique data, no workflow ownership, and no hard-to-switch integrations.

As base models become cheaper and better, wrapper startups face three pressures:

• Foundation model providers ship similar features natively.
• Enterprises build internal versions using APIs.
• Incumbents bundle equivalent capabilities into products customers already pay for.

A simple gut check: if you rip the LLM out of your product and little proprietary value remains, your moat is thin.

4. LLM adapters: incumbents playing defense and offense

Then there are incumbents that already own distribution and workflows and are now infusing them with AI---Microsoft Copilot in Office, Salesforce Einstein in CRM, Adobe Firefly in design, GitHub Copilot for developers.

In the short term, these players face real challenges: high inference costs, customers expecting AI to be "free," and competitive pressure that makes price increases difficult. Margins compress. Founders often dismiss them with, "What they ship sucks."

Over time, however, incumbents can:

• Bundle AI into existing products.
• Increase ARPU via premium tiers.
• Reduce churn by making core products meaningfully better.
• Optimize costs through routing, fine-tuning, and scale.

Because they already own the customer, the data, and the workflow, incumbents are often better positioned than startups to capture AI value---even if they move slowly and their first iterations are mediocre.

AI-native challengers still have a path, but usually only when they reimagine the workflow entirely, not when they add features incumbents can copy and bundle.

5. A simple moat test for AI startups

Across the AI stack, a few truths are becoming clear:

• AI capability itself is not scarce.
• Moats are not stationary; as models improve, moats move and the value chain shifts.
• Durability comes from ownership, not feature novelty.

A VC suggested a simple stress test: If base models become dramatically cheaper and more capable next year, what happens to your company?

If the honest answer is, "Customers can't leave without real operational pain, regulatory risk, or migration cost," there may be a moat.

If the answer is, "We'll ship more features," the moat is likely very thin.

6. Where does structural advantage sit in the AI stack?

Competition is fierce at every layer. Every software company is now rebranded as an AI company, and it's increasingly difficult to separate signal from noise. What follows is my opinion as a framework for thinking about the sector, not investment advice or a recommendation to buy or sell any security. A useful lens is to ask which players own more of the stack versus assembling it from others.

One company worth studying through that lens is Google, which has built across several layers of the AI stack, in some cases long before AI was fashionable. That vertical integration is a genuine structural strength. It also carries real risks of its own, which I cover below, so this is a study in trade-offs rather than a verdict on any stock.

Silicon: Google built before it was cool

Google introduced TPUs in 2016---not because GPUs were scarce, but because they were suboptimal for Google's workloads: search ranking, ads, and translation.

TPUs were internal-only for years. Google didn't monetize them until much later. That's telling. This wasn't a "we need AI exposure" move; it was an operational necessity.

Contrast that with companies scrambling for NVIDIA allocation in 2023--2024.

Even today, many Google workloads do not depend on NVIDIA at all, insulating the company from GPU pricing cycles. TPUs are tightly co-designed with TensorFlow, compiler stacks, networking, and Google's data-center topology.

Power and data centers: AI as a first-class workload

Google began optimizing power, cooling, and placement for machine learning over a decade ago. DeepMind-driven optimizations reportedly reduced cooling energy usage by roughly 40%.

This is recursive advantage: AI improving the infrastructure that runs AI.

Compare that with companies only now discovering that power---not GPUs---is the binding constraint, and hyperscalers signing rushed nuclear or gas contracts under pressure. Google didn't panic-buy power. They designed for it.

Models: inventing the stack, then giving it away

Google's AI paradox is that it invented much of the modern stack, published it, and then appeared to have gotten "outgunned" commercially.

Transformers, BERT, T5, PaLM---all originated internally. Even today, GPT-style models rest heavily on Google-published research.

The critique that "Google missed the ChatGPT moment" misses the structural point. Google optimized for defensive integration across search, ads, and Android. Moreover, Google probably didn't want to cannibalize their search engine franchise.

In contrast, OpenAI optimized for distribution via novelty. Those are very different games.

Distribution: AI as a default, not a product

Google's biggest moat is boring and ubiquitous: Search, Gmail, Docs, Android, Chrome, YouTube.

For Google to be successful, Gemini doesn't need to be the best model. It needs to be good enough, cheap enough, and ubiquitous.

That's classic bundling power. AI becomes table stakes, and marginal inference cost is amortized across massive ad and subscription revenue. One could argue that Google is among the few players whose marginal cost of inference could approach zero. If AI does become table stakes, that could position it well as a low-cost supplier of inference, though whether any of this translates into shareholder returns is a separate and far-from-certain question.

The risks to this thesis

Owning the stack is a strength, but it is not a guarantee of good outcomes, and the same integration cuts both ways:

• Regulatory and antitrust risk: Google faces active antitrust proceedings, and potential remedies affecting search, ad-tech, or default-distribution deals could weaken the very bundling advantage described above.
• Cannibalization risk: generative answers can erode the high-margin search-and-ads franchise that funds everything else, so success in AI could come at the expense of the core business.
• Execution and competition risk: owning the layers does not guarantee shipping the best products, and well-capitalized rivals and open-source models can close gaps quickly.
• Valuation and expectations risk: structural advantages can already be reflected in the price; a strong business is not automatically a strong investment.

None of these are predictions. They are reminders that any single-company thesis, including this one, can be wrong.

Buying into AI versus building it

Other players have taken a different route: spending cash, raising debt, or pulling in massive private capital to buy relevance.

Microsoft is the strongest example. Its partnership with OpenAI was a brilliant distribution move---but it's still not vertical integration.

Microsoft didn't build GPT. It wrote a ~$13B check and secured exclusive cloud hosting. Microsoft still depends heavily on NVIDIA GPUs, doesn't fully control OpenAI's roadmap, and faces margin pressure because inference costs don't vanish at scale. This is partnership leverage, not ownership.

Meta offers another contrast. Despite heavy investment, it has not (so far) dominated AI infrastructure or models in the way some expected. Instead, it open-sourced LLaMA---effectively choosing ecosystem gravity over cloud economics. In my view that's a smart defensive move, but it's still vertical mainly at the application layer.

Legacy players like IBM and Oracle face a harder road on AI infrastructure, in my opinion. They bring real strengths, deep enterprise relationships, large installed bases, established distribution, and substantial cash flow, but on AI specifically they have announced partnerships, rebranded products, and promised transformations while competing with less research depth, infrastructure, and tooling than the vertically integrated players. A common playbook among firms in this position is familiar:

• Acquire AI startups
• Rebrand existing products
• Sell "AI-powered" features without changing workflows

Where that's all a company does, the risk is brief novelty, incremental revenue, and thin pricing power as bundled alternatives appear elsewhere. That's my read, not a prediction, and any of these companies could execute better than I expect.

Why Google is structurally different

Compared to everyone else, Google stands apart:

• No acquisitions to buy relevance
• No loading up on debt to buy its place in the value chain
• No dependency on third-party models
• No need for circular financing to get news headlines
• No reliance on short-term monetization optics
• Willingness to wait years for compounding advantage from their AI investments

It looked slow---until it wasn't!

To be clear, that structural picture is an observation, not a stock call. The risks laid out above are real, a lot of good news may already be reflected in the price, and a durable business does not automatically make a good investment. Reasonable, informed people can look at the same facts and reach the opposite conclusion.

---

Rubric Advisors, LLC is an investment adviser registered with the State of California DFPI. Registration does not imply a certain level of skill or training. This article is general and educational, not personalized investment advice, and is not a recommendation to buy or sell any security. Any companies mentioned are for illustration only and do not represent holdings or recommendations. Past performance does not guarantee future results. See our Disclosures and Form ADV for additional important information.