BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//132.216.98.100//NONSGML kigkonsult.se iCalcreator 2.20.4//
BEGIN:VEVENT
UID:20261013T072614EDT-7839bAK8dW@132.216.98.100
DTSTAMP:20261013T112614Z
DESCRIPTION:PAC-Bayesian Generalizations Bounds for Deep Neural Networks\n
 \nOne of the defining properties of deep learning is that models are chose
 n to have many more parameters than available training data. In light of t
 his capacity for overfitting\, it is remarkable that simple algorithms lik
 e SGD reliably return solutions with low test error. One roadblock to expl
 aining these phenomena in terms of implicit regularization\, structural pr
 operties of the solution\, and/or easiness of the data is that many learni
 ng bounds are quantitatively vacuous when applied to networks learned by S
 GD in this 'deep learning' regime. Logically\, in order to explain general
 ization\, we need nonvacuous bounds. We return to an idea by Langford and 
 Caruana (2001)\, who used PAC-Bayes bounds to compute nonvacuous numerical
  bounds on generalization error for stochastic two-layer two-hidden-unit n
 eural networks via a sensitivity analysis. By optimizing the PAC-Bayes bou
 nd directly\, we are able to extend their approach and obtain nonvacuous g
 eneralization bounds for deep stochastic neural network classifiers with m
 illions of parameters trained on only tens of thousands of examples. We co
 nnect our findings to recent and old work on flat minima and MDL-based exp
 lanations of generalization.\n\n \n\nTime permitting\, I will discuss rece
 nt work on computing even tighter generalization bounds associated with a 
 learning algorithm introduced by Chaudhari et al. (2017)\, called Entropy-
 SGD. We show that Entropy-SGD indirectly optimizes a PAC-Bayes bound\, but
  does so by optimizing the 'prior' term\, violating the hypothesis that th
 e prior be independent of the data. We show how to fix this defect using d
 ifferential privacy. The result is a new PAC-Bayes bound for data-dependen
 t priors\, which we show\, up to some approximations\, delivers even tight
 er generalization bounds. Joint work with Gintare Karolina Dziugaite\, bas
 ed on https://arxiv.org/abs/1703.11008\n\n \n\n \n
DTSTART:20171110T203000Z
DTEND:20171110T213000Z
LOCATION:Room 1205\, Burnside Hall\, CA\, QC\, Montreal\, H3A 0B9\, 805 rue
  Sherbrooke Ouest
SUMMARY:Daniel Roy (University of Toronto)
URL:https://www.mcgill.ca/mathstat/channels/event/daniel-roy-university-tor
 onto-282448
END:VEVENT
END:VCALENDAR