Can you guys help me clarify something? To my understanding, if you want to give an agent more knowledge, you use some vector based retrieve system and package up a prompt with that context. Fine-tuning is for a more customizable format for your outputs. Is that correct or can I fine-tune more information right into the model? Thanks!

Following

All I can find online is how to do it, not why

Hi! RAG works very well as a first-stab at context customization. For more specialized situations, it often becomes better to get more hands-on and fine-tune the generalized model itself to get a more specialized model. There are a lot of variables involved (resources, problem complexity, data, etc.) in making a decision, but hope that helps as a starting point.

@Ethan Shaotran Any articles on this topic?

Fine tuning is meant to improve the performance of a model at a certain task, and that's the type of evaluation criteria that has been used for fine tuning/reinforcement learning techniques in nearly all research done thus far that I've seen. The techniques are not meant to "insert information" into a model reliably, and theory wise the closest you can get is increasing the statistical likelihood of outputting some token given the set of tokens thus far (assuming we're talking about an autoregressive model). Practically speaking maybe it's possible to push that aforementioned statistical likelihood so high for all of the data that you want the model to learn that it's effectively "learned" it, e.g. 99.999% of the time will spit out the information you're looking for when asked about it. I think your best bet would be to find out if anyone has run a study on this before - maybe come through some academic papers.

Thanks, @Daniel Hsu. I think we're on the same page. I'm jumping into fine-tuning Llama 2 and wanted to make sure I had my theory straight.

@Chris Johnston I'm also trying to finetune Llama2/Falcon for specific usecases. Any pointers/references you can share regarding data prep before starting finetuning?

I've been focusing on text embedding retrieval over fine-tuning. Only thing I know about training language models is they are easy to "burn", ie overtrain. I fine-tune image models and the technique is turn the learning rate very low and train for a long time... but does that translate to LLMs... 🤷‍♂️

I’m skewing towards finetuning on specific knowledge base and using retrieval for continuous data updates. This seems to work decently well for our usecase. To make it more conversational we had to finetune again on a instruction dataset specific to our use-case.