Robust Deep Learning

These three weeks on robust deep learning are part of the MVA course: theoretical foundations of deep learning : https://www.math.univ-toulouse.fr/~fmalgouy/enseignement/indexMva.html

• Week 1: Bayesian Models and Uncertainty
• Week 2: Bayesian Neural Networks
• Week 3: Applications of Deep Learning Robustness

The deadline for sending the reports is December 16 2024, 5pm. Please send a single archive file (to nicolas.thome at isir.upmc.fr) containing:

• A report answering the questions below with experimental results and their analyses. PDF format requested.

• The source codes (.ipynb) of your work

Here are the specific elements to be specified in the reports:

Week 1:

• Bayesian linear regression: results of the predictive distribution on the synthetic dataset [Question 1.4]

• Theoretical analysis to explain the form of the distribution (simplified case \(\alpha=0\), \(\beta=1\)) [Question 1.5]

• Non-linear regression: analysis of the Gaussian basis feature maps results [Question 2.4/2.5]

Week 2:

• Commente Laplace’s approximation results [Question 1.2]

• Part I.3 « Variational inference » : comment the class LinearVariational. What is the main difference between Laplace’s and VI’s approximations?

• MC dropout results: analyse predictive distribution on the 2-moons dataset [Question 2.1]. What is the main difference between MCdropout and the VI approximation in part I.3?

Week 3:

• Comment results for investigating most uncertain vs confident samples [I.1]

• Failure precition:

  • Explain the goal of failure prediction

  • Comment the code of the LeNetConfidNet class [II.1]

  • Analyze results between MCP, MCDropout and ConfidNet [II.2]

• OOD detection: analyse results and explain the difference between the 3 methods [III.1]