The latest batch of language models can be much smaller yet achieve GPT-3 like performance by being able to query a database or search the web for information. A key indication is that building larger and larger models is not the only way to improve performance. This video provides a gentle intro RETRO, DeepMind’s retrieval-augmented Transformer. There are more details in the associated blog post:
Corrections:
1:15 - *knowledge-intensive tasks
---
Contents:
Introduction (0:00)
Retrieval-enhanced language models (0:40)
The Illustrated Retrieval Transformer (1:55)
World-knowledge vs. language knowledge (2:57)
The database and how to query it and retrieve information (4:45)
Feeding the prompt and retrieved information into the model (5:39)
The architecture of the model (6:00)
---
Paper:
Authors: Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Millican, George van den Driessche, Jean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, Diego de Las Casas, Aurelia Guy, Jacob Menick, Roman Ring, Tom Hennigan, Saffron Huang, Loren Maggiore, Chris Jones, Albin Cassirer, Andy Brock, Michela Paganini, Geoffrey Irving, Oriol Vinyals, Simon Osindero, Karen Simonyan, Jack W. Rae, Erich Elsen, Laurent Sifre
----
Twitter:
Blog:
Mailing List:
---
More videos by Jay:
Experience Grounds Language: Improving language models beyond the world of text
Language Processing with BERT: The 3 Minute Intro (Deep learning for NLP)
Explainable AI Cheat Sheet - Five Key Categories
The Narrated Transformer Language Model
4 views
4
0
9 months ago 01:47:19 1
Unlock the Ultimate Escape: 2-Hour AI Video DJ set Downtempo Psychedelic Trip [mixed by escapall]
9 months ago 00:17:57 1
Dr. Joe Dispenza: Control THIS Powerful INVISIBLE FORCE and SUCCESS will Follow! (ft AI)
9 months ago 00:03:37 1
Alan Walker, Sabrina Carpenter & Farruko - On My Way
9 months ago 00:03:59 1
Freddie Mercury sings Toxicity by System of a Down (AI Cover)
9 months ago 00:04:20 1
HoneyComeBear - Achillea
9 months ago 02:08:36 1
“STRING OF FIRE“ Pure Dramatic 🌟 Most Powerful Violin Fierce Orchestral Strings Music
9 months ago 00:05:31 1
Nights of Malice - Purging The Slime [NEW 2023]
9 months ago 00:05:40 1
INVIRULANT - THE DEATH OF $LAM (RISE OF THE INVIRULANTS) [OFFICIAL MUSIC VIDEO] (2023) SW EXCLUSIVE
9 months ago 00:05:45 2
Нейрофотошоп от
9 months ago 00:01:59 1
Moonlit Solitude | Gothic Symphonic Metal | AI Music Video | Element 14
9 months ago 00:01:49 1
4k LookBook. Panties & Bras Intimate Lingerie. Montage Of The Most Beautiful Ladies. AI Art #90
9 months ago 00:01:06 1
AI lookbook | Sexy nurse girls | 섹시 간호사 룩북
9 months ago 00:10:15 1
Fall of the Crystal Empire - MLP Fan Animation
9 months ago 01:09:52 1
SAMURAI GIRL - 80’s Synthwave Music / Synthpop / Chillwave / Cyberpunk Mix (Study, Work & Gaming)
9 months ago 00:08:45 1
STOP Using Midjourney, This FREE AI Can Make You $7,530/month Instead!
9 months ago 00:06:48 1
Slipknot - Yen [OFFICIAL VIDEO]
9 months ago 00:03:45 1
Cocaine • Joe Robinson • JJ Cale | Eric Clapton Cover
9 months ago 00:12:54 1
Time: Do the past, present, and future exist all at once? | Big Think