Avoiding Catastrophe: Active Dendrites Enable Multi-Task Learning in Dynamic Environments (Review)
#multitasklearning #biology #neuralnetworks
Catastrophic forgetting is a big problem in mutli-task and continual learning. Gradients of different objectives tend to conflict, and new tasks tend to override past knowledge. In biological neural networks, each neuron carries a complex network of dendrites that mitigate such forgetting by recognizing the context of an input signal. This paper introduces Active Dendrites, which carries over the principle of context-sensitive gating by dendrites into the deep learning world. Various experiments show the benefit in combatting catastrophic forgetting, while preserving sparsity and limited parameter counts.
OUTLINE:
0:00 - Introduction
1:20 - Paper Overview
3:15 - Catastrophic forgetting in continuous and multi-task learning
9:30 - Dendrites in biological neurons
16:55 - Sparse representations in biology
18:35 - Active dendrites in deep learning
34:15 - Experiments on multi-task learning
39:00 - Experiments in continual learning and adaptive prototyping
49:20 - Anal
4 views
9
0
1 month ago 00:08:04 1
Catastrophe in France: maddened rivers wash away everything
2 months ago 00:00:00 18
đź”´ Hurricane Milton Live Cams - Major Storm Surge in Florida
2 months ago 00:30:25 3
Dystopian Futures of Astronomical Suffering | Documentary about S-risks and Longtermism
2 months ago 00:09:02 1
Hurricane Helene devastates the southern United States! The world is praying for people!
2 months ago 00:04:45 1
Nature’s Fury: Kathmandu Battles Massive Floods and Landslides, Nepal!
2 months ago 00:05:55 1
Artist Françoise Gilot
3 months ago 00:16:38 1
Essential Functions of a Small Business: Creating An Organisational Structure For Your Business
3 months ago 00:08:05 1
TORMENT FROM HEAVEN FELL ON MEDINA! Storm and floods in Saudi Arabia.
3 months ago 00:08:02 1
GERMANY IN RUINS! Nature is angry at almost the whole country.
4 months ago 00:01:44 1
Ready or Not: Home Invasion - Official Gameplay Trailer
4 months ago 00:08:09 1
DESERTS BECOME RIVERS! Floods in Saudi Arabia, Jazan. Heavy rains in KSA.
4 months ago 00:29:33 1
Innovating to zero! | Bill Gates
4 months ago 00:37:54 1
Ukraine Struggling, EU in Crisis, and Trump Ready to Shake Things Up! | Davis Pyne
4 months ago 00:03:45 1
OUR HOLLOW, OUR HOME - Burial Season (OFFICIAL VIDEO)
5 months ago 00:05:41 1
FLOODS Like TSUNAMI! Turkey UNDER WATER! HOMES & CARS Washed Away in HORRIFIC Floods in Ordu, Turkey
6 months ago 01:01:51 1
Tragic! Shocking Train Crashes Collisions and Mistakes Caught on Camera You Must See to Believe
6 months ago 00:00:11 1
Blasting in quarry
6 months ago 00:39:34 1
Putin’s Deadly Serious Warning to NATO | David T. Pyne
6 months ago 06:45:28 1
LIVE News | Hungarian Leader Orbán Sounds Alarm on Nuclear Risks in Ukraine Conflict
6 months ago 00:01:01 1
Go-Go Town! release date trailer - PC Gaming Show 2024
6 months ago 00:00:15 1
Surface Mine Blasting
6 months ago 00:52:01 1
Nyiragongo, one of the world’s most active and deadly volcanos, could erupt at any time
6 months ago 01:02:10 1
Tragic! Best Of Semi Truck Crashes Moments Filmed Seconds Before Disaster That Will Shock You
7 months ago 00:11:57 1
Foggia Italy my initial walk through.. Not so good. AVOID THIS ONE! - Foggia Italy - ECTV