Single Headed Attention RNN: Stop Thinking With Your Head with Stephen Merity - #325

Today we’re joined by Stephen Merity, startup founder and independent researcher, with a focus on NLP and Deep Learning. In our conversation, we discuss: Stephen’s newest paper, Single Headed Attention RNN: Stop Thinking With Your Head. His motivations behind writing the paper; the fact that NLP research has been recently dominated by the use of transformer models, and the fact that these models are not the most accessible/trainable for broad use. The architecture of transformers models. How Stephen d