Mean-field theory of learning dynamics in deep neural networks

Abstract

Learning dynamics of deep neural networks is complex. While previous approaches made advances in mathematical analyses of the dynamics of two-layer neural networks, addressing deeper networks have been challenging. In this talk, I will present a mean field theory of the learning dynamics of deep networks in the feature-learning regime and discuss its implications for practice.

References:

https://arxiv.org/abs/2205.09653

https://arxiv.org/abs/2305.18411

https://arxiv.org/abs/2309.16620

Speaker Intro

Cengiz (pronounced "Jenghiz") comes to Harvard SEAS from the Flatiron Institute's Center for Computational Biology (CCB), where he was a a research scientist in the neuroscience group. Before CCB, Cengiz was a postdoctoral associate at Janelia Research Campus, and before that a Swartz Fellow at Harvard. Cengiz received a doctorate in physics from Brown University and undergraduate degrees in physics and electrical engineering from Bogazici University. He is a native of Tosya, Turkey.