Andrej Karpathy Building makemore Part 5: Building a WaveNet
🎯 Загружено автоматически через бота:
🚫 Оригинал видео:
📺 Данное видео принадлежит каналу «Andrej Karpathy» (@AndrejKarpathy). Оно представлено в нашем сообществе исключительно в информационных, научных, образовательных или культурных целях. Наше сообщество не утверждает никаких прав на данное видео. Пожалуйста, поддержите автора, посетив его оригинальный канал.
✉️ Если у вас есть претензии к авторским правам на данное видео, пожалуйста, свяжитесь с нами по почте support@, и мы немедленно удалим его.
📃 Оригинальное описание:
We take the 2-layer MLP from previous video and make it deeper with a tree-like structure, arriving at a convolutional neural network architecture similar to the WaveNet (2016) from DeepMind. In the WaveNet paper, the same hierarchical architecture is implemented more efficiently using causal dilated convolutions (not yet covered). Along the way we get a better sense of and what it is and how it works under the hood, and what a typical deep learning development process looks like (a lot of reading of documentation, keeping track of multidimensional tensor shapes, moving between jupyter notebooks and repository code, ...).
Links:
makemore on github:
jupyter notebook I built in this video:
collab notebook:
my website:
my twitter:
our Discord channel:
Supplementary links:
WaveNet 2016 from DeepMind
Bengio et al. 2003 MLP LM
Chapters:
intro
intro
starter code walkthrough
let’s fix the learning rate plot
pytorchifying our code: layers, containers, , fun bugs
implementing wavenet
overview: WaveNet
dataset bump the context size to 8
re-running baseline code on block_size 8
implementing WaveNet
training the WaveNet: first pass
fixing batchnorm1d bug
re-training WaveNet with bug fix
scaling up our WaveNet
conclusions
experimental harness
WaveNet but with “dilated causal convolutions”
the development process of building deep neural nets
going forward
improve on my loss! how far can we improve a WaveNet on this data?
1 view
0
0
1 month ago 04:01:25 11
[Andrej Karpathy] Let’s reproduce GPT-2 (124M)
1 month ago 00:27:13 2
But what is a GPT? Visual intro to transformers | Chapter 5, Deep Learning
1 month ago 00:44:17 1
No Priors Ep. 80 | With Andrej Karpathy from OpenAI and Tesla
2 months ago 00:18:11 53
НОВОСТИ ИИ: Подписка на ChatGPT за 2000$
2 months ago 00:26:10 1
Attention in transformers, visually explained | Chapter 6, Deep Learning
2 months ago 00:06:53 1
Elon Musk says losers use LiDAR. [Explanation video]
3 months ago 00:47:27 1
Projeto Secreto da OpenAI: Descubra as Últimas Inovações da IA e Fique Super Atualizado no IA News#6
3 months ago 00:07:41 1
Educação 100% com IA, FBI invade celular, Hype das IA no fim, e muito mais
6 months ago 00:40:08 1
The Most Important Algorithm in Machine Learning
6 months ago 00:08:29 1
CATL’s sodium hybrid battery will be 30% cheaper & revolutionise the world
6 months ago 00:08:55 1
Tesla reveals timelline for massive electric Semi production at $ factory
8 months ago 00:26:53 1
Vedal & Neuro Build A Language Model From Scratch
8 months ago 00:03:19 2
How To Study Hard - Richard Feynman
8 months ago 00:16:39 1
Phi-1: A ’Textbook’ Model
9 months ago 00:20:13 1
GPT-5: Everything You Need to Know So Far
11 months ago 00:14:07 1
“Что в имени тебе моем?“ Учимся генерировать новые имена у звездного разработчика Tesla и OpenAI.
11 months ago 00:59:48 21
Введение в большие языковые модели от Andrej Karpathy
11 months ago 00:03:51 1
ČENDEŠ - Karpaty, Karpaty [OFFICIAL 2017 4K]
12 months ago 00:30:09 1
Andrej Karpathy - AI for Full-Self Driving at Tesla
12 months ago 01:03:42 3
[SafeCode Live] ML в AppSec
1 year ago 00:05:48 3
Advice for machine learning beginners | Andrej Karpathy and Lex Fridman
1 year ago 03:28:47 25
Andrej Karpathy Tesla AI, Self-Driving, Optimus, Aliens, and AGI Lex Fridman Podcast #333
1 year ago 03:08:46 1
George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God | Lex Fridman Podcast #387
1 year ago 00:07:56 11
[Lex Clips] How to hack the simulation | Andrej Karpathy and Lex Fridman