Digital images are ubiquitous : from professional and smartphone cameras to remote sensing and medical imaging, technology steadily improves, allowing to obtain ever more accurate images under ever more extreme acquisition conditions (shorter exposures, low light imaging, finer resolution, indirect computational imaging methods, to name a few).
This course introduces inverse problems in imaging (aka image restoration), namely the mathematical models and algorithms that allow to obtain high quality images from partial, indirect or noisy observa- tions. After a short introduction of the physical modeling of image acquisition systems, we introduce the mathematical and computational tools required to achieve that goal. The course is structured in two parts.
The first part deals with well-posed inverse problems where perfect reconstruction is possible under certain hypotheses. We first introduce the theory of continuous and discrete (fast) Fourier transforms, convolutions, and several versions of the Shannon sampling theorem, aliasing and the Gibbs effect. Then we review how imaging technology ensures the necessary band-limited hypothesis, and a few applications including: antialiasing and multi-image super-resolution, exact interpolation and registration for stereo vision, synthesis of stationary textures.
In the second part we deal with ill-posed inverse problems and the variational and Bayesian formulations, leading to regularized optimization problems (for posterior maximization) and to posterior sampling (not covered in this course). This part starts with a review of optimization algorithms including gradient descent, and the most simple splitting and proximal algorithms. Then we review increasingly powerful regularization techniques in historical order: from Wiener filters and Tikhonov regularization, to total variation, and non-local self-similarity. By the end of the course we briefly introduce an overture to recent approaches using pretrained denoisers as implicit regularizers of inverse problems via RED and plug and play algorithms for posterior maximization. The theory is illustrated by applications to image denoising, deblurring and inpainting.
Course contents
• Part 1 - Well-Posed inverse problems
– Discrete Fourier Transform 1D & 2D
– Shannon's Sampling Theorem - Exact interpolation
– Applications to remote sensing & subpixel stereo vision
• Part 2 - Ill-posed Inverse Problems
– Review of Probability & Statistics
– Tikhonov / Wiener Regularization
– Total Variation Regularization - Convex optimization
– Learning-based Regularization - Non-convex optimization
Prerequisites
Essential
• Vector Spaces (MAA206), Mathematical Analysis (MAA102, MAA202 or similar)
• Linear Algebra (MAA101), basic Python programming (CSE101)
Helpful but not strictly required
• Measure theory and Integration (MAA301),
• Numerical Linear Algebra (MAA208), advanced Python programming (CSE102, MAA106)
• Basic concepts of Probability and Statistics (MAA203, MAA204, MAA304 or MAA305)
Schedule
• Lecture 1 : Discrete Fourier Transform (1D) - PL/PE/LS/Q
• Lecture 2 : Discrete Fourier Transform (2D) - PL/LS/Q
• Lecture 3 : Shannon's Sampling Theorem - PL/Q
• Lecture 4 : Continuous Fourier Transforms - PL/LS/HW
• Lecture 5 : Probability & Statistics - PL/HW
• Lecture 6 : Tikhonov / Wiener / TV Regularization - Part 1 - PL/LS
• Lecture 7 : Tikhonov / Wiener / TV Regularization - Part 2 - PL/LS/HW
• Lecture 8 : Applications to remote sensing & subpixel stereo vision - PL/Q
• Lecture 9 : Learning-based Regularization - PL/LS/Q
Abbreviations:
PL = Plenary Lecture LS = Lab Session TD = Practical Exercises HW = Homework Assignment Q = Quizz
Evaluation
To validate this course, you will need to upload several homework assignments which count for the continuous assessment (CA) grade. This will be completed by a final exam (FE).
The ratio for the final grade is as follows: Final grade = 2/3 * CA + 1/3 * FE.
CA = Continuous Assessment is composed of
• 3 homework assignments (HW)
• Several Quizzes
FE = Final Exam at the end of the course (June 10th)
Final grade = 2/3 * CA + 1/3 * FE
The final exam will be composed of two parts:
• Part 1: Questions on the course ad practical exercises to be done on paper - no documents allowed
• Part 2: Computer code to complete in a Jupyter Notebook - bring your own laptop - all documents allowed
References
As a complement of the course materials (handout, slides, assignments) the following references can be useful.
J.M. Morel (2004), Cours de traitement de Signal et de l’Image, ENS Cachan, polycopié.
J.M. Bony (1994), Cours d’analyse - Théorie des distributions et analyse de Fourier, Ecole Polytechnique, polycopié.
C. Gasquet et P. Witomski (1995), Analyse de Fourier et applications, Masson.
S. Mallat (1997), A Wavelet Tour of Signal Processing, Academic Press.
N. Sabater, J.M. Morel and A. Almansa (2011). How Accurate Can Block Matches Be in Stereo Vision? SIAM Journal on Imaging Sciences, 4(1), 472–500.
Parikh, N., & Boyd, S. (2014). Proximal Algorithms. Foundations and Trends in Optimization, 1(3), 127–239.
A. Almansa, S. Durand, and B. Rougé (2004). Measuring and Improving Image Resolution by Adaptation of the Reciprocal Cell. Journal of Mathematical Imaging and Vision, 21(3), 235–279.
F. Malgouyres and F. Guichard (2001). Edge Direction Preserving Image Zooming: A Mathematical and Numerical Analysis. SIAM Journal on Numerical Analysis, 39(1), 1–37.
A. Buades, B. Coll and J.M. Morel (2006). A review of image denoising algorithms, with a new one. SIAM Multiscale Modeling and Simulation, 4(2), 490–530.
C. Barnes, E. Shechtman, A. Finkelstein and D.B. Goldman (2009). PatchMatch: a randomized correspondence algorithm for structural image editing. ACM Transactions on Graphics-TOG, 28(3), 24.
L. Raad, A. Davy, A. Desolneux and J.M. Morel (2017). A survey of exemplar-based texture synthesis. Annals of Mathematical Sciences and Applications, 3(1), 89–148.
Samuel Hurault. (2023). Méthodes plug-and-play convergentes pour la résolution de problèmes inverses en imagerie avec régularisation explicite, profonde et non-convexe. PhD thesis. Université de Bordeaux.
Ryu, E. K., Liu, J., Wang, S., Chen, X., Wang, Z., & Yin, W. (2019). Plug-and-Play Methods Provably Converge with Properly Trained Denoisers. In (ICML) International Conference on Machine Learning. Retrieved from http://proceedings.mlr.press/v97/ryu19a.html
R. Laumont, V. De Bortoli, A. Almansa, J. Delon, A. Durmus, and M. Pereyra (2023). On Maximum a Posteriori Estimation with Plug & Play Priors and Stochastic Gradient Descent. Journal of Mathematical Imaging and Vision, 65(1), 140–163.
