MadeStone

GPT (nanoGPT) from a beginner's perspective (Part 1)

Просмотры:
24
Дата загрузки:
03.12.2023 13:47
Длительность:
00:30:23
Категория:
Лайфстайл

Описание

In this video I went through Karpathy's nanoGPT Code base, explaining each line from a beginner's perspective.

nanoGPT is a character lever implementation of a GPT (Generative Pre-trained Transformer) referencing the official Transformer from the Attention is All You Need paper.

Karpathy is my role model in the field of AI Research, he is a cofounder of Open AI and former Director of AI at Tesla.

My GPT Practice Repo
https://github.com/Jaykef/GPT-Practice

References
nanoGPT: https://github.com/karpathy/nanoGPT
Attension is All You Need Paper: https://arxiv.org/pdf/1706.03762.pdf
The Illustrated Transformer: https://jalammar.github.io/illustrated-transformer/
The Annotated Transformer: http://nlp.seas.harvard.edu/2018/04/03/attention.html
Techniques for training large neural networks: https://openai.com/research/techniques-for-training-large-neural-networks
The State of GPT: https://karpathy.ai/stateofgpt.pdf

#gpt #nanogpt #karpthy #ai #nlp #llm #openai #google

Источники:

Рекомендуемые видео