The leaked Google memo and OpenAI's moats

Who has moats in AI?

I get it — lots of excitement abound with the “leaked internal Google memo” and tweets about "no moats" in the AI industry. But, I feel compelled to set the record straight with another AI obsessive megathread (which you are welcome to digest in thread form (over on Twitter @labenz).

TL;DR - We'll see everything, everywhere, all at once, but OpenAI (and Google) do have moats.

First, let's define what a moat is.  I asked Perplexity.ai and learned that Warren Buffett popularized the term, which refers to a competitive advantage that allows a company to maintain its market position and earn outsized profits.

So let’s do a brief visualization exercise to determine which companies might have earned a seat at the table. It’s 2025 and the major players in AI are meeting. Who’s there? Emad Mostaque for sure. But would you expect any of Sam Altman, Sundar Pichai, Satya Nadella, or Anthropic’s CEO Dr. Dario Amodei to be left out of this meeting?

I’d be shocked, and if you’d be too… that suggests moats.

Back to the leaked Google memo. There is a lot to like!  In particular, it begins with a great rundown of a handful of highly efficient techniques which do, in all likelihood, mean the beginning of real AI proliferation.

I also expect a lot of the specific predictions to come true – people will use open source models to evade the rules, to maintain control over data, and to have the opportunity to craft their own personal models over time. It's going to be wild!

I covered similar efficiency trends in this AI pricing analysis piece from Marginal Revolution January, which holds up reasonably well as AI content goes.

And I've also marveled at how much value OpenAI got from just 100K samples in their original RLHF paper, across 3 models, generating 100X parameter advantage. If anything, with the necessary software libraries coming online to power this, I'd say the memo underplays how much community-based RLHF'ing we're likely to see. Many corporations have the necessary data – think about all those recorded calls – and many communities can create data on this scale as well.

Research will also flourish on the base of open-sourced models, including mechanistic interpretability, which I personally consider the most promising path to safety.

But it does *not* follow from the recent reductions in prices, nor even from the proliferation of open-source RLHF models, that OpenAI will not be able to earn outsized profits

So, let us count the moats!

Moat 1. OpenAI's GPT-3.5-turbo is currently the best value in the utility LLM game. It's fast, cheap, easy to use, and reasonably effective, reliable, and safe. I don't see anyone positioned to meaningfully outperform or disrupt OpenAI on any of those dimensions in the immediate term. Why? Because despite the hype, open source models don't really hit on the same level, for reasons outlined here. So much of the "we matched ChatGPT" noise has in fact been people "running out to claim 2 mins of fame." Open-source models don't hit the same level of quality, as they lack the rigor that OpenAI shows in trying to convince themselves that their models are actually doing amazing.

Anthropic's Claude is the only worthy rival to GPT-3.5-turbo at present. At Athena, where I'm an AI advisor, we've found Claude to be more agreeable in tone, less prone to hallucinations, and better on safety in some domains. However, Anthropic plays it quiet so as to avoid feeding the hype cycle, but their GPT-4 equivalent is well on its way.

Another moat (2) that OpenAI has is that people in general are still very confused, unnerved, and even scared by AI. Many will want to use the safest, most established option, which currently happens to be OpenAI. They used to say “Nobody gets fired for going with IBM”; the modern echo might be "Nobody gets fired for going with OpenAI" This moat could easily be squandered – trust is hard to build & easy to break! – but I think OpenAI gets that, as shown by the 6 months of GPT-4 testing.

Moat 3: OpenAI has a top-notch product feedback loop and is collecting data at an unmatched scale via ChatGPT’s free tier. This means that GPT-3.5-turbo will continue to get better over time, which is not true for open-source models. Who wants to manage updates and vulnerability patches?

Onto pricing which is Moat 4. OpenAI has arguably led the market on price cuts, with a ~97% reduction over the last nine months. There's no reason to think they're done. At $2 per million tokens, how much do you really stand to save with open source? Open source can only undercut so much.

By being so aggressive on price so quickly, OpenAI is making it extremely hard for would-be LLM utility competitors to ever make a profit. In my opinion, people are underestimating how cheap AI is. GPT-3.5-turbo is only one cent for 5,000 tokens, which is equivalent to ten pages of writing. You can't read more than a dime's worth a day!

I have been studying recent news from MosaicML in preparation for an upcoming The Cognitive Revolution podcast episode. The company made headlines eight months ago when it introduced the GPT-3 model, and this week they have released several new products, including the Replit code model, an all-new inference platform, and a 65k token context window model.

While examining the pricing page, I noticed that the new MPT-7B-Instruct model is priced at exactly 25% of the GPT3.5-turbo, and the GPT-NeoX-20B is offered at the same price of $2/1 million tokens.

This tells me that even a company with as much talent and momentum as MosaicML cannot undercut OpenAI's volume product by all that much.

The truth is that for this business model to work, companies have to be generating orders of magnitude more tokens than the entire public internet has to date. Think about it this way: at GPT-3.5-turbo prices, you have to serve 100B tokens to pay for a single employee.  $200K revenue – 25% computing costs = maybe $150K gross profit. For this business model to work, you have to be generating orders of magnitude more tokens than the entire public Internet contains to date.  That’s a lot of tokens!

And good luck hiring ML PhDs for $150K these days – even amidst tech layoffs, OpenAI alums are recession-proof.

In the end, regardless of where you get it, low end commodity intelligence is going to be cheap, and most people will happily pay for the convenience of commercial utility AI. Just as I could grow my own food or generate my own electricity, but in practice don't.

Which brings me to Moat 5: privileged access to massive cloud compute Yes, you can run a 7B model on a laptop, and that does matter, but people already keep their email, documents, and photos in the cloud I expect LLMs to accelerate the shift to cloud. That’s why Anthropic partnered with Google, and why HuggingFace, champion of the open source movement, is also prioritizing a compute partnership with AWS.

Moat 6: GPT-4 – is not a small model. It is the only often-human-level AI on the market today, and people are happy to pay up for it. My company, Athenago, has a financially disciplined strategy but our AI strategy is to use GPT-4 first, maximize performance, and find cost savings later as necessary.

This strategy is shared by other companies as well. When I interviewed Flo Crivello of Lindy.ai, he said the same thing. GPT-4 is making life so much easier, and Moore's Law is on his side, so he's not worried about costs. Incredible excerpt from our interview here.

With leadership at both service companies and tech startups explicitly instructing teams not to worry about cost as they use GPT-4, OpenAI seems very well-positioned for outsize profits.

However, the main thing is that, for the first time in its history, OpenAI isn't telling how it made GPT-4. We don't know what data was used or how much, how many parameters or how much compute, how much user feedback was collected, or even if there might be undisclosed algorithmic advances at play.

We just know that it works on a qualitatively higher level, such that while randos are training Llama to imitate ChatGPT, OpenAI is using GPT-4 to characterize, monitor, refine, and extend itself. This is a GPT-4 feedback loop that is textbook moat. OpenAI gets this, of course, and it's always been against their Terms of Service to use their outputs to train competitive models, but now they are also taking product measures – no longer offering top token probabilities via the API – just to make it that much harder to attempt to "clone" GPT-4.

So, while OpenAI's hard-won practical knowledge will gradually diffuse, you can't win major market share by hiring people away and imitating. To win the high-end market away from OpenAI, companies have to not just match but beat them. Therefore, OpenAI's moat is still fairly wide.

Moat 7 then, is team and talent density. As someone who has interacted with dozens of OpenAI team members, I can tell you that Sam Altman is for real when he says it's the most talent-dense place he's been a part of.

It seems like the biggest moat is the challenge of assembling a team and building a culture that can actually push the AI frontier. There are a number of companies that can do it, and maybe one or two open source communities, but overall very few.

Google, btw, is definitely one of them. And the fact that Google also intends to stop publishing their results means that, outside of a few key places, people won't really know what's working on the frontier and will have to figure it out for themselves.

As an AI scout who always wants to know what's going on, that makes me sad. But it makes sense for them, and I think they are making this switch just in time for business purposes.

And btw if you think I’m writing this to endear myself to OpenAI by defending their honor, trust me when I say: they don’t care if you believe they have moats. and as one OpenAI team member memorably put it to me recently, “we have grand plans”

And that brings me to Moat 8: insane distribution and partnerships. Microsoft was first to productize a fine-tuned GPT-4, but that is just the tip of the iceberg.

For the next couple of years, OpenAI stands to earn outsize profits not by selling 1 trillion 3.5-turbo tokens / employee / year – which is what it would take! – but by selling best-available performance, delivered via "robust fine-tuning", on a "7-figure commitment" basis.

When nobody can match your quality at any price, you can set the entry point at $1.5M / yr, and plenty of major companies will pay up And indeed, OpenAI has announced one major customer after another. (To be clear, I don't know details of these deals).

And even as we speak, Bain consultants are systematically breaking jobs down into tasks, designing workflows that delegate tasks to AI, and gleefully calculating cost savings. I call this The Great Implementation; it’s a good time to be a consultant!

I can tell you from first-hand experience: for a great many tasks, this process works! GPT-4 cost is not negligible and latency currently an issue as well, but still savings and latency reduction are both 90%+ relative to human in most cases.

To be clear, this process doesn’t always reduce jobs – sometimes it serves to scale previously unscalable processes – and I’m not predicting the end of employment, but if we're honest, in many cases it definitely will.

Finally, Moat 9 is network effects. While AI doesn't seem to have the same network effects as web 2.0 / social media, it's still notable that every "Prompting 101" course, performance benchmark, library, and tool is built for / on / with OpenAI models first. This dynamic forces other utility LLM providers into a following position, having to match OpenAI as first requirement to compete.

Now… does all this mean that OpenAI is going to take all, as next week's The Cognitive Revolution podcast guest Robert Scoble worries they might? To me, this is going too far in the other direction, at least in the short term. (All bets off for 2025+!)

My best guess is that utility AI will look something like search, except this time it will be OpenAI / Microsoft leading the market, followed by Google Deepmind / Anthropic, which will likely find it hard to catch up in market share, despite similar performance.

In other words, Google will know how Bing has felt all this time, and maybe – just maybe – also vice versa.

Meanwhile, the open source ecosystem and all sorts of service providers will continue to flourish as well. Because *demand is exploding everywhere* and OpenAI can’t and doesn’t even want to do everything.

The biggest gap I see in OpenAI’s public product line right now is around fine-tuning. As of now, it does not look like OpenAI really wants to compete for low-end / small-model fine-tuning business.

The best model you can fine tune is the original davinci – that is the pre-Instruct, "world's biggest autocomplete" model And when you do, it costs $0.12 / 1000 tokens inference – >60X higher than GPT-3.5-turbo, and >240X MosaicML's new 7B model price.

Anecdotally, through summer/fall 2022, I was running a lot of fine-tuning and generally found that the OpenAI queue was super short. Around November of 2022, that seemed to change – we started to see queues of ~20 fine-tunings fairly often, especially during US business hours.

Today … best I can tell, that's holding steady, with 10-20 models often found in the queue. Low confidence data point here as I really don't have a ton of visibility, but this activity does not appear to have gone exponential.

This leaves a TON of opportunity for eg MosaicML to build businesses. as AI goes vertical across all verticals and more people begin to spend hours / day using AI products, customers will want cheap fine-tuned results for any number of reasons

For starters, many will want to train from scratch exclusively on proprietary data. Others will dream up all sorts of crazy use cases to take advantage of the shiny, new 65K context window.

Last thing: is Meta somehow winning? this galaxy brain take is where the memo really lost me. Meta is flexing their capabilities & endearing themselves to some in the research & open source communities, but they are taking needless risks and capturing limited value themselves.

As people start using models like Llama for eg spearphishing & propaganda, and Meta is once again revealed to have moved fast & broken stuff.

And contra the memo, I don’t think it really matters whose architecture the open source community is hacking on. As Neel Nanda would tell you, a lot of the details of the architectures themselves don’t seem to matter all that much anyway.

So… what have we learned? I don't psychologize others' AI takes, but I do think people would think more clearly about AI if they started thinking "both/and" instead of "either/or." Outcomes won't be binary; it's everything, everywhere, all at once I believe that OpenAI has several moats that will make it hard for competitors to catch up. While the recent reductions in prices and the proliferation of open-source models are exciting trends, they do not necessarily mean that OpenAI won't be able to earn outsized profits.

More concretely, the bottom lines: - Open source & proliferation in general will have profound impact - OpenAI & Google Deepmind / Anthropic have real moats - Because the market is exploding, many companies will succeed & grow - Meta is strong technically but def not winning.

I’d guess this will hold for a while, but tbh the whole technology paradigm could still turn on a dime. Especially as we shift to closed research, the next big breakthrough could come from anywhere, and may not be shared for a while. So keep your heads on a swivel!

That's it for today. As my dad would say: it you liked it, tell your friends; if you didn’t, tell your enemies :)

And subscribe to the @CogRev_Podcast on your preferred platform to hear me talk about all this with some of the smartest people in the world!