GPT (nanoGPT) from a beginner's perspective (Part 1)
Описание
In this video I went through Karpathy's nanoGPT Code base, explaining each line from a beginner's perspective.
nanoGPT is a character lever implementation of a GPT (Generative Pre-trained Transformer) referencing the official Transformer from the Attention is All You Need paper.
Karpathy is my role model in the field of AI Research, he is a cofounder of Open AI and former Director of AI at Tesla.
My GPT Practice Repo
https://github.com/Jaykef/GPT-Practice
References
nanoGPT: https://github.com/karpathy/nanoGPT
Attension is All You Need Paper: https://arxiv.org/pdf/1706.03762.pdf
The Illustrated Transformer: https://jalammar.github.io/illustrated-transformer/
The Annotated Transformer: http://nlp.seas.harvard.edu/2018/04/03/attention.html
Techniques for training large neural networks: https://openai.com/research/techniques-for-training-large-neural-networks
The State of GPT: https://karpathy.ai/stateofgpt.pdf
#gpt #nanogpt #karpthy #ai #nlp #llm #openai #google
Рекомендуемые видео


















