https://cerebralvalley.ai logo
#06-technical-discussion
Title
# 06-technical-discussion
d

Don Alvarez

04/19/2023, 4:54 PM
I'm seeing a lot of buzz around pinecone, in part because it's used in BabyAGI. When I look at BabyAGI, though, it looks like pinecone is really only being used as little more than a simple key-value store. I get the theoretical advances of a vector store for use with LLM's, but are there any actual practical advantages of vector stores like pinecone for a BabyAGI-like use case compared to a more conventional MongoDB or Redis cache kind of store?
g

Gerred Dillon

04/19/2023, 4:56 PM
you can totally use Redis's vector similarity to get started, or literally just shove your vectors into an np.array, or look at pgvector
d

Don Alvarez

04/19/2023, 5:00 PM
I guess my question is even more naive than that - is there a reason in a BabyAGI like system to not just stash a vanilla json blob of the conversation in a store and leave it at that? (As opposed to using vector stores)
g

Gerred Dillon

04/19/2023, 5:00 PM
OH, sure. there's reasons you may or may not want to or need to turn it into vectors. i.e. if you have a doc that uses cat and you query on "kitty"
there's NO need to use a vector DB at all. it solves a specific set of use cases
d

Don Alvarez

04/19/2023, 5:02 PM
I think you're saying basically what I was seeing, that vector DB's are very cool but BabyAGI isn't actually doing anything with that vector functionality - for that use case it's just a glorified generic key value store
g

Gerred Dillon

04/19/2023, 5:06 PM
Yeah, exactly. Don't worry about it until you need it, and I'm usually gauging that on...basically, did it improve response quality (by virtue of adding more correct info to my context window?)
h

Han

04/19/2023, 5:10 PM
No there’s zero advantage.
You can store everything in a numpy array stack and it’s going to perform better and faster
Or you could pip install hnswlib and shove everything in memory
d

Don Alvarez

04/19/2023, 5:15 PM
I've always been a big fan of Actor-based systems like Microsoft Orleans (similar to temporal.io and Akka, or as a way to write Elixir-like systems in C#). The question about pinecone and vector DB's came up while I was trying to wrap my head around whether actor-based systems are viable architectures for LLM experiments. I'm feeling like the answer to the actor based question is more "it depends" than "no", which is making me happy.
c

Chris Alexiuk

04/19/2023, 5:25 PM
Yeah, and honestly if you need the Vector DB, there are plenty of lightweight and local systems you can use like Chroma. Pinecone is like bringing a nuke to a knife fight.
h

Han

04/19/2023, 5:26 PM
Chroma is hnswlib + DuckDB.
2
IMG_4236.jpg
t

Ted Cheng

04/19/2023, 6:00 PM
if you read closely in the BabyAGI original commit. the context agent (which calls pinecone) was actually not even invoked. I think there's a lot of hype in the space right now and people are excited to jump on new frameworks or tooling. In reality, Pinecone has a nice clean interface, so if you need a vector store, it's not bad to start with it. However, stating with np.array is probably just fine. If you are larger scale, you might want to a more traditional solution like postgres + pgvector extension, which has its own operational benefits.
h

Han

04/19/2023, 6:07 PM
pgvector performs really poorly at relevancy/recall.
It’s information retrieval, so tools like Elastic/OpenSearch, Vespa, Solr, or other in-house stacks will run supreme at large scale.
c

Chris Alexiuk

04/19/2023, 8:15 PM
Something like BabyAGI/AutoGPT are definitely overhyped.
1