> In other words, how does the LLM actually "know" it's supposed to spit back an answer in the first place?
Instruction tuning, as far as I can tell. The original "Alpaca" model took the LLaMA base and fine-tuned it with question-answer type content. From that, the generation went from prose-leaning to Q&A type responses.
Instruction tuning, as far as I can tell. The original "Alpaca" model took the LLaMA base and fine-tuned it with question-answer type content. From that, the generation went from prose-leaning to Q&A type responses.