Where graph databases live in a future AI data stack
Where graph databases live in a future AI data stack
Another once specialized database technology from the previous big data era might find its way into AI products thanks to retrieval augmented generation.
If 2024 is supposed to be the year of “AI in production,” the first step to getting there is throwing as many ideas at the problem first and hoping for the best on the other side.
We already saw that with the emergence of retrieval augmented generation, or RAG, which at first seemed like a bandaid but now seems like a permanent fixture. But even those new permanent fixtures have limitations, and there’s still a bar of quality (and practicality) for all these newer model-powered products to cross.
Now, another big data technology may be getting a closer look as these kinds of applications move closer to production and demand higher-quality or more accurate responses: graph databases.
More specifically, some companies are exploring implementing graph search into a RAG setup to improve the relevancy of the results. Like many other once highly specialized technologies, such as vector databases, graph databases potentially offer another layer of relevancy to deliver the right information to a prompt and improve the quality (or accuracy) of the results.
“You can think of graph independently and vector independently as one way to store features—graph captures the connections much better than key-value, and vectors do the semantic capturing much better in a more condense format,” Naren Narenden, chief scientist at Aerospike, a provider of graph and vector databases, told me. “Independently they work well as new kinds of feature stores, but you can see how they can come together. You can have a RAG application that starts out with semantic search that identifies a few key nodes as a first step, and use the graph to gather the rest of the information.”
The idea is pretty straightforward: instead of using just a graph database, or a just vector database, why not both?
Developers use the same kind of classic RAG approach to retrieve information, but in this case, it’s retrieving nodes—a given point with a lot of information connected to other points—and then running a much more intelligent graph search.
Graph databases and graph search was already in use in cases where one entity—say, an e-commerce customer or financial transaction—has a lot of attributes and is intricately linked to many other concepts. You could see a new transaction show up (which would include location, card number, vendor, amount, and so on) and quickly chase down a lot of information about it to determine if it is an anomaly and should be flagged as fraud.
That includes a process called traversal, where you search for information by literally move from node to node to find what you’re looking for. And that’s largely limited based on where you start on the graph in the first place. But taking a piece of information coming in and using the classic RAG playbook—embedding it, searching for similar results, and spitting out relevant nodes—gives you more ideas of where to start on the graph.
And it turns out it helps potentially alleviate two of the major problems each type has: you have more ideas of where to start search on a graph; and you get a better understanding of relationships and connections than you might get in a vector database.
“Vector spaces are completely opaque—it’s a bunch of numbers, and human beings can’t parse that,” Emil Eifrem, CEO of graph database provider Neo4j, told me. “Graph spaces meanwhile are completely explicit. I show an apple and a tennis ball, and vector similarity search will say they’re similar but not tell us why. It’s some dimensions in some kind of latent space out there. In graph space, though, an apple and an orange we see are related explicitly because they’re both fruit.”
For now, it’s still a two-step process for those providers, and it doesn’t show up all that often. But as companies move from proof-of-concept to actual production and understand the real performance needs of these apps, anything and everything is up for discussion. And that includes determining whether graph databases fit into a more modern AI data stack.
The return of graph databases
Perhaps the most well-known startup in graph databases historically is Neo4j, which was founded in 2007. Neo4j last raised $325 million at a valuation over $2 billion in June 2021, and said in January that it had passed $100 million in annual recurring revenue. Neo4j is backed by GV, Greenbridge Venture Partners, Eurazeo, One Peak, and several others. (And ironically, like many older big data startups Neo4j had fallen so far off the radar of most that I talk to that pretty much everyone was curious what they’re up to.)
Graph databases have typically found a place in financial services, particularly around fraud prevention, as well as recommendation engines. Each person or transaction has some kind of unique information that is intricately linked to other pieces of information. In the case of a recommendation engine, it could be a user’s prior purchasing or search behavior, which can change in near real time.
Vector databases, meanwhile, are well suited for that flood of unstructured information. Lots of different kinds of data sources—from PDFs to Slack messages or confluence pages—can all get dumped into the same place for retrieval. Graph thrives on understanding those relationships, and as a result, needs something that falls in the middle.
Graph databases have certainly not gone away, though they have been traditionally thrown into the bucket of “very important for some use cases.” And that’s still going to be the case going forward, as the technical barrier for implementing this dual setup is pretty high. Still, this month the International Organization for Standardization actually published GQL, a sibling language to SQL designed for graph databases.
Sponsored
This week’s newsletter is brought to you by Felicis
Felicis has been investing in an impressive variety of AI and infrastructure companies, including: Runway, Weights & Biases, MotherDuck, Supabase, Metaplane, Vannevar Labs, poolside, and Flower Labs. A generalist firm started by Google’s first product manager, Felicis is known for supporting founders with their Founder’s Pledge.
Read what founders say about working with Felicis.