I'm seeing a lot of buzz around pinecone, in part because it's used in BabyAGI. When I look at BabyAGI, though, it looks like pinecone is really only being used as little more than a simple key-value store. I get the theoretical advances of a vector store for use with LLM's, but are there any actual practical advantages of vector stores like pinecone for a BabyAGI-like use case compared to a more conventional MongoDB or Redis cache kind of store?

you can totally use Redis's vector similarity to get started, or literally just shove your vectors into an np.array, or look at pgvector

I guess my question is even more naive than that - is there a reason in a BabyAGI like system to not just stash a vanilla json blob of the conversation in a store and leave it at that? (As opposed to using vector stores)

OH, sure. there's reasons you may or may not want to or need to turn it into vectors. i.e. if you have a doc that uses cat and you query on "kitty"

I think you're saying basically what I was seeing, that vector DB's are very cool but BabyAGI isn't actually doing anything with that vector functionality - for that use case it's just a glorified generic key value store

Yeah, exactly. Don't worry about it until you need it, and I'm usually gauging that on...basically, did it improve response quality (by virtue of adding more correct info to my context window?)

No there’s zero advantage.

I've always been a big fan of Actor-based systems like Microsoft Orleans (similar to temporal.io and Akka, or as a way to write Elixir-like systems in C#). The question about pinecone and vector DB's came up while I was trying to wrap my head around whether actor-based systems are viable architectures for LLM experiments. I'm feeling like the answer to the actor based question is more "it depends" than "no", which is making me happy.

Yeah, and honestly if you need the Vector DB, there are plenty of lightweight and local systems you can use like Chroma. Pinecone is like bringing a nuke to a knife fight.

Chroma is hnswlib + DuckDB.

if you read closely in the BabyAGI original commit. the context agent (which calls pinecone) was actually not even invoked. I think there's a lot of hype in the space right now and people are excited to jump on new frameworks or tooling. In reality, Pinecone has a nice clean interface, so if you need a vector store, it's not bad to start with it. However, stating with np.array is probably just fine. If you are larger scale, you might want to a more traditional solution like postgres + pgvector extension, which has its own operational benefits.

pgvector performs really poorly at relevancy/recall.

Something like BabyAGI/AutoGPT are definitely overhyped.