Chapter 23 sections from Deep Learning with PyTorch.
8 items
A large language model is trained in two broad phases. The first phase is pretraining.
Scaling laws describe how model performance changes as we increase compute, parameter count, dataset size, and training tokens.
Pretraining teaches a language model to predict text. It does not directly teach the model to follow user instructions, answer safely, maintain dialogue structure, or format outputs in a useful way.
Instruction tuning teaches a model to imitate demonstrations.
Reinforcement learning from human feedback improves model behavior using preference data. However, collecting large amounts of human feedback is expensive, slow, and difficult to scale consistently.
Large language models can often perform new tasks without updating their parameters.
A language model becomes more useful when it can interact with external systems.
Retrieval-augmented generation, usually abbreviated RAG, combines a language model with an external information retrieval system.