Value-Based Deep RL Requires Explicit Regularization

Presented at ICML 2021 workshop on Overparameterization: Pitfalls and Opportunities (Spotlight) and the workshop on RL for Real Life Workshop (Spotlight). If you are interested in how implicit bias affects representations in deep Q-learning algorithms, consider checking out our previous work: Implicit Under-Parameterization Inhibits Data-Efficient Deep Reinforcement Learning (Aviral Kumar*, Rishabh Agarwal*, Dibya Ghosh, Sergey Levine)