Robust Deep Learning

These three weeks on robust deep learning are part of the MVA course: theoretical foundations of deep learning : https://www.math.univ-toulouse.fr/~fmalgouy/enseignement/indexMva.html

The deadline for sending the reports is November 23 2023, 5pm. Please send a single archive file (to nicolas.thome at isir.upmc.fr) containing:

  • A report answering the questions below with experimental results and their analyses. PDF format requested.
  • The source codes (.ipynb) of your work

Here are the specific elements to be specified in the reports:

Week 1:

  • Bayesian linear regression: results of the predictive distribution on the synthetic dataset [Question 1.4]
  • Theoretical analysis to explain the form of the distribution (simplified case \(\alpha=0\), \(\beta=1\)) [Question 1.5]
  • Non-linear regression: analysis of the Gaussian basis feature maps results [Question 2.4/2.5]

Week 2:

  • Commente Laplace’s approximation results [Question 1.2]
  • Part I.3 « Variational inference » : comment the class LinearVariational. What is the main difference between Laplace’s and VI’s approximations?
  • MC dropout results: analyse predictive distribution on the 2-moons dataset [Question 2.1]. What is the main difference between MCdropout and the VI approximation in part I.3?

Week 3:

  • Comment results for investigating most uncertain vs confident samples [I.1]
  • Failure precition:
    • Explain the goal of failure prediction
    • Comment the code of the LeNetConfidNet class [II.1]
    • Analyze results between MCP, MCDropout and ConfidNet [II.2]
  • OOD detection: analyse results and explain the difference between the 3 methods [III.1]