Lecture 4: Transfer Learning and Transformers (Full Stack Deep Learning - Spring 2021)

In this video, you will learn about the origin of transfer learning in computer vision, its application in NLP in the form of embedding, NLP’s ImageNet moment, and the Transformers model families. 00:00 - Introduction 00:42 - Transfer Learning in Computer Vision 04:00 - Embeddings and Language Models 10:09 - NLP’s ImageNet moment: ELMO and ULMFit on datasets like SQuAD, SNLI, and GLUE 16:49 - Rise of Transformers 18:20 - Attention in Detail: (Masked) Self-Attention, Positional Encoding, and Layer Normalization 27:33 - Transformers Variants: BERT, GPT/GPT-2/GPT-3, DistillBERT, T5, etc. 36:20 - GPT3 Demos 42:53 - Future Directions