Hi, I want to build an LLM to be able to assess quality of a content (blog,video,audio) for a learner and be able to generate questions from the content. Any hints to go faster ?

Hi Bill - what have you tried so far? And what does the assessment of quality involve?

Parallelize it.

What i va tried so far is to give a piece of text ( from transcript video, blog text) to chatgpt and ask it to generate questions @Rishabh Bhargava

Maybe I didn’t get the context correctly, but which step are you trying to speed up? The indexing or the actual question answering?

ah effectively, I was not precise enough. My question was rather to know if there were frameworks or examples which would enable me not to start from scratch.

Don’t think you need frameworks to do it at all? It’s just grab text, use OpenAI api to generate questions based on the individual texts? As with every other workflow, you can write out the steps as a DAG. Or, in the LLM world, chains.

@Bill Metangmo - seems like there are a few things that might be helpful. In terms of frameworks for working with LLMs, I'm sure you've seen/played with Langchain (although you can get quite far with just calling the LLMs directly). In terms of improving the quality of question generation or assessing quality of content, that's where it can get tricky. Usually, people try some number of prompts on a small reference dataset until they're happy with the quality of generations. Quality of generation goes up the more clearly you're able to describe what you want from the LLM, and the more examples you provide. Of course, this just boils down to understanding your problem domain really well.

@Kyle Casterline is good at quickly solving such problems, perhaps he can advise you something.