Do you want a Vector Database when a Document Database is all you need? In RAG, you pass relevant context to the LLM as an additional part of the prompt, which raises the question - how do you find relevant context? It turns out LLMs are only one of the models to come out of the Generative AI revolution - but there is a slightly lesser-known sibling, the Embeddings Model. An Embeddings Model takes text and embeds it in a high dimensional space, effectively creating a vector representation of that text. If done well, texts with similar meanings will have similar vectors, and unrelated texts will have unrelated vectors. Vector similarity can be numerically calculated (for example with cosine similarity), meaning we are able to determine which texts are most relevant (or similar) to each other. When the number of vectors you are searching through becomes large enough, this is usually done with approximate knn, an algorithm finds the k most similar vectors without needing to calculate the distance of each vector individually. With RAG, where we now have the tools to take all of our possible context documents, generate vector embeddings for them, store those vectors in a vector database, and then find the most relevant vectors. But the LLM prompt needs the context as text, not as an embedding vector, so logically you also store the documents in the vector database - and herein lies the crux of the problem. What RAG actually needs is not a Vector Database, but a Document Database. Specifically, that Document Database needs to support Vector Search (sometimes referred to as a “Vector Index”) - meaning it can run approximate knn (or similar) across vector fields. Read more: https://www.redactive.ai/blog/you-dont-want-a-vector-database

I'm confused. Isn't RAG a superset of this? From what I understand. RAG is Document Database + embedding of request + using LLM to pick the best of the top 10 results of the Document Database?

The arguement being made here is that the standalone vectordb doesnt really make sense "Specifically, that Document Database needs to support Vector Search (sometimes referred to as a “Vector Index”) - meaning it can run approximate knn (or similar) across vector fields." people are looking to do search and are often rebuilding the whole stack with little utility from doing so.