YC
Mathematical Approaches to Image Processing with Carola Schönlieb
@ Y Combinator
05/10/18
Takeaways
CS
Carola Schönlieb
05/10/18
@ Y Combinator
I started my research in mathematics in Vienna, focusing on partial differential equations, which model various natural phenomena in physics and biology.
CS
Carola Schönlieb
05/10/18
@ Y Combinator
My first paper was on the Cahn-Hilliard equation, which describes phase separation in metallic alloys when cooled to a certain temperature.
CS
Carola Schönlieb
05/10/18
@ Y Combinator
The stability analysis of solutions to the Cahn-Hilliard equation helps understand how systems react to perturbations, particularly in the context of creating alloys.
CS
Carola Schönlieb
05/10/18
@ Y Combinator
During my research, I discovered that the Cahn-Hilliard equation was used for image restoration, which involves replacing damaged parts of a digital image with suggested content from surrounding areas.
CS
Carola Schönlieb
05/10/18
@ Y Combinator
The technique used for image restoration is similar to the content-aware fill feature in Photoshop, but it predates Photoshop's development.
CS
Carola Schönlieb
05/10/18
@ Y Combinator
My PhD focused on image restoration, specifically using differential equations to solve problems similar to those addressed by Photoshop's content-aware fill.
CS
Carola Schönlieb
05/10/18
@ Y Combinator
In my postdoc, I shifted towards inverse imaging problems, where the observed data is not a direct image but a transform, such as in CT scans.
CS
Carola Schönlieb
05/10/18
@ Y Combinator
In CT imaging, what is measured are projections of a 3D object, and the goal is to reconstruct the object from these projections.
CS
Carola Schönlieb
05/10/18
@ Y Combinator
The challenge in CT imaging is that the data collected is often insufficient for high-resolution images, leading to noise in the measurements.
CS
Carola Schönlieb
05/10/18
@ Y Combinator
Denoising is integrated into the image reconstruction process, addressing both the lack of data and the noise present in measurements.
CS
Carola Schönlieb
05/10/18
@ Y Combinator
Successful image denoising methods focus on preserving edges, which are crucial for maintaining the integrity of the image's features.
CS
Carola Schönlieb
05/10/18
@ Y Combinator
Total variation regularization is a widely used technique in image denoising that helps preserve sharp discontinuities in images.
CS
Carola Schönlieb
05/10/18
@ Y Combinator
While handcrafted models for image denoising are still relevant, deep neural networks are increasingly outperforming them in various scenarios.
CS
Carola Schönlieb
05/10/18
@ Y Combinator
Neural networks can struggle with images they haven't been trained on, highlighting the continued importance of handcrafted models in certain contexts.
CS
Carola Schönlieb
05/10/18
@ Y Combinator
The integration of handcrafted models with neural networks presents exciting opportunities for mathematicians to explore and improve image denoising techniques.
CS
Carola Schönlieb
05/10/18
@ Y Combinator
Understanding the properties of handcrafted models can help inform the development of neural networks, particularly in terms of stability and performance.
CS
Carola Schönlieb
05/10/18
@ Y Combinator
My recent work involves starting with handcrafted models and parameterizing them in a simplified way, rather than using millions of parameters. I learn parameters from actual examples that I want my handcrafted model to produce, which is known as parameter estimation or optimization.
CS
Carola Schönlieb
05/10/18
@ Y Combinator
The motivation for using parameter estimation in image processing has shifted towards machine learning, which has generated more interest in this area.
CS
Carola Schönlieb
05/10/18
@ Y Combinator
Even with handcrafted models, you can still understand and prove things about your solution, ensuring guarantees on the results.
CS
Carola Schönlieb
05/10/18
@ Y Combinator
In using neural networks for problems like computer tomography, we are exploring how to combine handcrafted models with neural networks, particularly in how we feed them prior information.
CS
Carola Schönlieb
05/10/18
@ Y Combinator
When reconstructing 3D objects from line integrals, it's important to remember that the measurements are line integrals of the object you want to reconstruct.
CS
Carola Schönlieb
05/10/18
@ Y Combinator
In a sequential manner, we adapt our objective while processing images, rather than feeding all images at once, which allows for better optimization.
CS
Carola Schönlieb
05/10/18
@ Y Combinator
Neural networks often do not need to solve optimization problems exactly; instead, they minimize loss over a finite set of training examples, which is only an approximation of the larger problem.
CS
Carola Schönlieb
05/10/18
@ Y Combinator
The goal in training a neural network for denoising images is to minimize the least squares error between the denoised and clean images across the training set.
CS
Carola Schönlieb
05/10/18
@ Y Combinator
To generalize well, you might not want to minimize the loss exactly over the training set, as this could lead to overfitting and poor performance on unseen data.
CS
Carola Schönlieb
05/10/18
@ Y Combinator
In collaborations with hospitals, we focus on developing algorithms that maximize the quality of high-resolution images from limited data, particularly in medical imaging.
CS
Carola Schönlieb
05/10/18
@ Y Combinator
We are working with various disciplines, including chemical engineering, to understand dynamic processes through imaging techniques.
CS
Carola Schönlieb
05/10/18
@ Y Combinator
In plant sciences, we use aerial and hyperspectral imaging to monitor forest health and identify invasive species by analyzing the spectral properties of trees.
CS
Carola Schönlieb
05/10/18
@ Y Combinator
Lidar measurements provide 3D models of trees, allowing for more detailed analysis compared to traditional planar images.
CS
Carola Schönlieb
05/10/18
@ Y Combinator
The idea of virtual restoration allows us to create digital copies of historical manuscripts without physically altering them, preserving their original state.
CS
Carola Schönlieb
05/10/18
@ Y Combinator
In a recent exhibition, we showcased a manuscript alongside its virtual restoration, demonstrating how technology can aid in preserving cultural heritage.
CS
Carola Schönlieb
05/10/18
@ Y Combinator
For those interested in image processing, I recommend starting with foundational books and exploring recent research from institutions like UCLA.