Build a Large Language Model (From Scratch)
Sebastian Raschka
Code-along videos for Sebastian Raschka’s book of the same name. The seven videos set up the environment, prepare text data, code the attention mechanisms and a GPT model, pretrain it, and then finetune it for classification and for following instructions. The code is free on GitHub; the book is sold separately.
What you'll learn
- Tokenizing text and building data loaders
- Coding self-attention and multi-head attention
- Implementing a GPT model in PyTorch
- Pretraining on unlabeled text
- Finetuning for text classification
- Instruction finetuning
