AI in November: Embeddings take off and consumption models bounce back
The most consequential tech story in decades could spark an even bigger race among startups to chart the future of AI.
Happy post-Thanksgiving to the U.S. crowd here! Some stuff seems to have happened in November that was surprising and has semi-resolved. And surprisingly, based on the sample-size-of-one that is my calendar, we seem to be heading into a very busy December.
November saw one of the biggest, weirdest, and most consequential stories of the past decade. I’m not going to bother doing a full recap of Sam Altman’s brief ouster at OpenAI here, but needless to say the oddly structured board didn’t provide a great answer to why they fired him and the entire company almost followed in protest.
But Altman’s ouster and the chaos that followed had several secondary effects in AI—particularly providing a massive opening for startups and competitors while the largest company in AI was in complete disarray.
That obviously wasn’t the whole story in AI in November, though. So here’s what we’ll be covering from what happened in November, inadvertently one of the most consequential months in tech history.
Startups pounce on OpenAI’s chaos: multiple startups and companies dropped a series of well-timed launches that all seem to have the ability to supplant at least one—or more—parts of its business. That includes companies like Perplexity, AI21, Together AI, and Inflection. But there are clearly some concerns among OpenAI customers amid a very delicate transition to the production phase of AI.
Embeddings take off: While some of the larger companies—namely Amazon and Google—offered competing embedding models to OpenAI, the field was pretty sparse. That has changed radically in the last 45 days, and we’re starting to see one of the most important parts of the AI stack morph into a very competitive marketplace.
Consumption models rebound: After more than a year of hand-wringing around consumption-based pricing getting hit by macroeconomic forces, we seem to be poised for a major rebound. That’s to the enormous benefit of the data abstraction layers like Snowflake and Databricks, but it also has a lot of consequences for AI going forward as a signal for demand.
With that, let’s get to it!
The embeddings market suddenly ignites
It’s time to graduate the market for embeddings models from “heating up” to a full-on competitive race.
Amid all the chaos of OpenAI, one of the more important arcs in AI development in November was the sudden explosion in challenger embedding models. OpenAI had largely enjoyed a very long honeymoon period in AI with ada-002 and no dramatic competitor to speak of—until things got very busy, very quickly, in November.
Retrieval augmented generation (or RAG) is increasingly becoming a go-to method to combat hallucinations, which remains a thorn in the side of language model usage. And high-performance embeddings models—at as low of a price as you can get—are a key part of building out a strong RAG architecture. There’s the option to fine-tune models with custom data to try to reduce (extremely confident) wrong answers from prompts, but it’s more time and resource intensive and generally requires some expertise. RAG splits the difference.
On the embedding model front, we got three (or four, depending on who you talk to) major launches in the last 45 days or so:
Cohere launched an updated version of its embedding model, which outperforms ada-002 on a variety of benchmarks.
Voyage AI, a startup dedicated to embedding models, launched its embeddings model and includes the Stanford trio of AI experts Fei-Fei Li, Chris Manning, and Chris Ré as advisors. (Technically this was October 29, but that’s basically November.)