SAVE THE DATE
Friday morning at 11 am
Workshop GT Réseaux Profonds et Représentations Distribuées
Location : Paris Sud University, building 660 - Orsay - Shannon amphitheatre
Title: Over-Parametrization in Deep Learning
Presented by : Levent Sagun
Stochastic gradient descent (SGD) works surprisingly well in optimizing the loss functions that arise in deep learning. However, it is unclear what makes SGD so special? In this talk, we will discuss the role of over-parametrization in deep learning as an attempt to understand what's special in SGD. In particular, we will see empirical results that show that in certain regimes SGD may not be so special at all. We will discuss whether we can explain this by looking at the geometry of the loss surface. To this end, we will take a look at the Hessian of the loss function and its spectrum, and see how increasing the number of parameters may lead to an easier optimization problem.
All information on this webpage : GT Réseaux Profonds & Représentations Distribuées