Zeta Alpha’s Trends in AI — February 2022. ConvNets comeback, Neural IR, Multimodal

In this edition of the Zeta Alpha monthly webinar, we highlight news and research at the start of February 2022. Timestamps: 00:00 — Intro 2:20 — News: prominent startups funding 5:48 — Facebook’s new supercomputer 6:52 — PyTorch’s 5th anniversary 8:10 — OpenAI’s embeddings API, alignment, DeepMind’s competitive coding Language Models, IBM Watson’s death 12:15 — The Zeta Alpha platform 14:01 — AutoRL, Language Models as Zero-shot planners 18:31 — Language Models Retrieval 19:20 — Conversational AI: LaMDA 25:00 — Benchmarking knowledge-intensive LMs: The Web is Your Oyster. 29:12 — Language Models as a Service 33:30 — Text and Code Embeddings by Contrastive Pre-Training 44:05 — ConvNets Comeback against Vision Transformers 48:45 — Multimodal Models 51:15 — Training Ultra Large Models: Mixture of Experts 56:60 — Open Source 20B parameters GPT: GPT-NeoX-20B 57:50 AlphaCode