Has anyone worked with LlamaIndex (https://gpt-index.readthedocs.io/en/latest/index.html#) but for non-text data? My use case is around building searchable indexes around observability data that is collected by the organizations. For eg, lot of companies use flame graphs, or distributed traces, which are inherently hierarchical in nature. It is also in some sense a sequence data and could be thought of as a sentence. I wanted to understand how I can modify LLamaIndex for some of the data we’ve collected on our cluster.

Might be better to think of LLamaIndex as a text-only index, and to build an abstraction that lets an LLM agent query it as a “tool”. Ex: 1. You have a “docs index” which the Agent uses to query for docs, and it is backed by a LLamaIndex SimpleVectorIndex 2. You have a “traces index” which the agent uses to query for relevant traces, and transforms the traces into text which can be passed back to the Agent as context for Q/A (assuming your goal is simple Q/A for this example)

Oh wow! These are comprehensive, I’ll check it out and ask questions!!

You’re welcome, I am in deep haha

Awesome @Vishwanath Seshagiri @Conner Swann! I shared this question with Jerry Liu (we’re old colleagues) and he said better support for semi-structured data is on their immediate roadmap, but he’s happy to chat with either of you to brainstorm ideas. lemme know if you want to connect!

Hey! Jeffrey, thanks a lot for reaching out to him! I'd love to connect with him, if there is a chance! 🙂

Jerry is a great guy! You’d learn a lot from him @Vishwanath Seshagiri

That sounds amazing! Can't wait to meet with him! 🙂