How ChatGPT works (non tech explainer)

In this video I use an analogy between ChatGPT and predictive text to help explain in everyday terms how ChatGPT works. ChatGPT and similar large language models (LLMs) at times have amazing abilities to respond to questions that people ask, seeming to behave as if they understand what is being asked of them. But what is the nature of the AI that we are engaging with? By using the analogy with predictive text I show some of the possible limitations of this current generation of transformer neural networks. ▬▬ Chapters ▬▬ 00:00 - I’m late! 00:14 - Sentence from predictive text 01:16 - Asking ChatGPT about my analogy 02:02 - How does predictive text work? 03:25 - Scaling up to ChatGPT 05:23 - Chat, quality and safety 05:53 - As if it understands 06:35 - Limitations of ChatGPT 09:49 - Understanding? Cognition? Society? ▬▬ Other videos about ChatGPT ▬▬ If you want to know more about the “sophisticated safety and quality guardrails“, here is a more techy video that describes the additional training that was applied to the underlying language model in order to get ChatGPT to be better aligned for the chat bot task from the default behaviour of the pre-trained GPT language models: How ChatGPT is Trained by Ari Seff ▬▬ References ▬▬ The word frequency data about ’control’ usually following ’remote’ was checked here: The “Attention Is All You Need“ paper on arXiv: Reference and links for the underlying data used to estimate that human are exposed to in the order of 100s of millions of words as we develop into young adults: A Longitudinal Investigation of the Role of Quantity and Quality of Child-Directed Speech in Vocabulary Development And here is a discussion (with lots of further useful links) about a key early piece of research in this area: New Research Ignites Debate on the ‘30 Million Word Gap’ For my video I simply used the rough, upper estimate from this early study that some children by age 4 have been exposed to around 45 million words and therefore simply extrapolate this into teenage years to get the order of magnitude of 100s of millions of words by young adulthood. ▬▬ Audio visual sources ▬▬ All clips of ChatGPT are original recordings. Diagrams were simply created on PowerPoint. Additional video clips from Storyblocks: