Has anyone taken the plunge on fine tuning any of ...
# 06-technical-discussionc
Clay Smith
08/13/2023, 11:32 PMHas anyone taken the plunge on fine tuning any of the LLaMA 2 models? Working my way through that using the Huggingface “autotrainer” … and have many questions.
👀 1
Clay Smith
08/13/2023, 11:34 PMshould have searched first! https://cerebral-valley.slack.com/archives/C04SZTQU84R/p1691088344656879
Clay Smith
08/13/2023, 11:36 PM@umar ıgan All the examples I’m seeing are for QA fine tuning… is it possible to fine tune with general knowledge, or is the idea you embed facts in answers, etc?
a
Ankush Agarwal
08/14/2023, 1:18 AMI’m starting to go down this rabbit hole. I want to train it on a general set of documents. will share my findings as i go
u
umar ıgan
08/14/2023, 8:01 AMNot necessarily @Clay Smith the reason of fine tuning on QA dataset is to make a domain specific LLM.
here is an example of more generalist approach;
https://colab.research.google.com/drive/1Zmaceu65d7w4Tcd-cfnZRb6k_Tcv2b8g?usp=sharing
2 Views