10 years of NLP history explained in 50 concepts | From Word2Vec, RNNs to GPT
From RNNs to Transformers to GPT-4, the leap in intelligence in Deep Learning research for Language Modelling and NLP has been a steady and educational growth. In this video, I explain 50 concepts that cover the basics of NLP like Tokenization and Word Embeddings, to seminal work like RNNs, Seq2Seq, Attention, to innovative Transformer models like BERT, GPT, XL-Net, and InstructGPT. I present the challenges we have faced in previous designs, and what the current architectures do to improve it, as well as highlight areas of improvement for future research.
For an overview of multimodal models that combine text with other input modalities like images, videos, audio, check out:
The video is divided broadly into 5 chapters, each adding up to 50 concepts that chronologically cover the advancements of NLP.
0:00 - Intro
0:47 - Basics of Language Modelling
1:44 - RNNs, Seq2Seq, Encoder-Decoders
6:18 - Understanding Transformers
8:26 - LLMs - BERT, GPT, XLNet, T5
12:54 - Human Alignment, ChatGPT, GPT4
16:20 - Outro
Thanks for watching!
#deeplearning #machinelearning #gpt #ai
Papers referenced:
Word2Vec:
GRUs:
LSTMS:
Seq2Seq:
Enc-Dec Attention:
Attention is All You Need:
GPT:
BERT:
Relative Position Embeddings:
Transformer-XL:
T5:
XL-Net:
Performer:
Linformer:
Longfromer:
LORA:
GPT-3:
InstructGPT:
GPT-4 Technical Report:
1 view
818
226
4 months ago 00:03:37 1
The Kiffness x Mozart Cat - Piano Playing Cat (Remix)
4 months ago 10:00:01 1
🔥 Cozy Fireplace 4K UHD! Fireplace with Crackling Fire Sounds. Christmas Fireplace Ambience 2024
4 months ago 00:02:33 1
Cross Ethnic Harmonica Jamming - Sarah X Siri
4 months ago 00:08:04 1
Delicious Bread For Breakfast, I’ve been doing it for 20 years, it has never misled me!
4 months ago 00:06:53 1
York - Injection (Chill Out Mix) (From: York - Islanders)
4 months ago 01:11:38 1
Uuskalvinismin nousu ja romahdus
4 months ago 00:14:56 1
9 вещей, которые ВСЕГДА делают БЕДНЫЕ и НИКОГДА не делают БОГАТЫЕ
4 months ago 01:21:24 1
“World War 3 Begins in just 51 Days!“ Western media WARNS it is coming in Mid-Summer! | Redacted
4 months ago 00:45:05 1
«Техническая эстетика» Документальный фильм об Уральской школе дизайна
4 months ago 00:45:33 1
Rotting Christ - Pro Xristou - (Full album 2024)
4 months ago 00:13:48 1
Как выбрать 7 книг, которые изменят вашу жизнь?
4 months ago 00:51:01 1
• GROND - Howling From The Deep [Full-length Album] Old School Death Metal
4 months ago 00:58:30 1
Live Replay 🔴 UCI Mountain Bike Eliminator World Cup 2024 | Round 4, Sakarya - Türkiye
4 months ago 00:01:50 1
The new 911. The one and always.
4 months ago 00:25:18 1
[4K] – Beautiful Los Angeles Landing – Alaska Airlines – Airbus A321-200N – LAX – N929VA – SCS 1174
4 months ago 02:01:34 1
Melodic Progressive House Mix: “The Beauty Within“ (myni8hte pres. Hopeful Skies 190)
4 months ago 00:12:53 1
“FBI IS FRAMING TRUMP AGAIN“ (Rogan Left SPEECHLESS By Evidence Of Manipulation In Trump Case)
4 months ago 00:19:19 1
Omnec Onec - Core Teachings of the Venusians
4 months ago 00:59:29 1
Burning Sun: Exposing the secret K-pop chat groups - BBC World Service Documentaries
4 months ago 00:19:09 1
TOP 10 | INCREDIBLE ADELE Covers in The Voice
4 months ago 00:57:46 1
Green Day live @ Ohibò Club 2013 | Milan, Italy (Most Complete Show) [05/22/2013]