Lecture 8 part 2: Deep Neural Networks

This is Lecture 8 - part 2 - of the KT EP3260 Fundamentals of Machine Learning over Networks (MLoNs). This lecture reviews the fundamentals and recent advances of deep neural networks (DNNs). In particular, this lecture covers its non-convex optimization landscape, various algorithms to address it, back propagation, preconditioning the optimization landscape, adaptive step size (including ADAM, RMSprop, and ADGRAD), and batch normalization. It then addresses learning and inference over networks, where we ma