I want to improve my retrieval with my vector database (in my case, PGVector). Sometimes, it comes back to some unrelated articles . My plan is too have an LLM look at the results and decide if its actually a good result but I'm afraid it will hurt speed. Has anyone tried to do something like this - AI filtered retrieval results?

What you're asking is re-ranking the results based on relevance. LLMs cannot do this because they need to be explicitly trained on relevance. On a side note: If you're getting very irrelvant results, you might want to look into the quality and granularity of the embeddings that has been generated. Also what's the number of documents that you're retrieving for each query?

I think it could judge relevance. just ask is, does this answer the question. For instance, I'm asking "What team is Cristina Escalante on?" which is only answered in 1 article - "About Cristina Escalante". I'm getting back that but also things shes just mentioned in. I don't want articles that say "if you get stuck, ask Cristina Escalante". I'm also getting "About Priscilla de Roode Torres".. I'm guessing because she has a spanish name. Cristina isn't mentioned at all. Ask GPT, "does this article answer the following question: { Question }. Return a boolean in the JSON in the following format: { isAnswered: BOOLEAN} then check. if false, eliminate it from the final prompt out to GPT.

yeah - dm'ed you