Sign in to like videos,
subscribe, and earn!
Languages
No language preferences
Interests
No interest preferences
How earnings work →
Got more questions? View FAQs
© 2026 LYKSTAGE
Show more
Adit Roy 
0 subscribers
Uploaded to LYKSTAGE
Jul 8, 2026
We build a Generatively Pretrained Transformer (GPT), following the paper "Attention is All You Need" and OpenAI's GPT-2 / GPT-3. We talk about connections to ChatGPT, which has taken the world by storm. We watch GitHub Copilot, itself a GPT, help us write a GPT (meta :D!) . I recommend people watch the earlier makemore videos to get comfortable with the autoregressive language modeling framework and basics of tensors and PyTorch nn, which we take for granted in this video. Links: - Google colab for the video: https://colab.research.google.com/drive/1JMLa53HDuA-i7ZBmqV7ZnA3c_fvtXnx-?usp=sharing - GitHub repo for the video: https://github.com/karpathy/ng-video-lecture - Playlist of the whole Zero to Hero series so far: https://www.youtube.com/watch?v=VMj-3S1tku0&list=PLAqhIrjkxbuWI23v9cThsA9GvCAUhRvKZ - nanoGPT repo: https://github.com/ Com deep learning neural network language model pytorch gpt generatively chatgpt attention is all you need attention
Read more
Similar videos
