Welcome to the staging ground for new communities! Each proposal has a description in the "Descriptions" category and a body of questions and answers in "Incubator Q&A". You can ask questions (and get answers, we hope!) right away, and start new proposals.
Are you here to participate in a specific proposal? Click on the proposal tag (with the dark outline) to see only posts about that proposal and not all of the others that are in progress. Tags are at the bottom of each post.
How does fine tuning work? Question
LLMs are trained on a lot of text. At its heart, an LLM is a glorified autocomplete engine. You give it text, and it adds more text that matches the pattern.
You need to "direct" the autocomplete, so that it's useful. For example, when given a question, you don't want it to simply keep rephrasing the question and saying how confused it is in different ways. You want it to complete with an answer. I understand that this part is accomplished by fine tuning. I know that models are provided a set of QA pairs, but also these are a much smaller set than the training corpus. Fine tuning also does alignment, where the model is taught to give useful and "good" answers.
How exactly does fine tuning work, at a high level? I am asking only for an overview of common/popular techniques, used by models like GPT, Claude, Bard, Llama and their open source relatives. I don't need a comprehensive list of every single technique - I understand it is a complex topic and there's a lot of variation. However, I would like enough mention of technical detail, that I can follow up on individual subtopics in my future research.
If there is a good, accessible summary available in the literature (like a review paper), feel free to mention it! However, I would expect an answer that at least summarizes or paraphrases it, rather than a link-only answer.
0 answers
0 comment threads