--- /dev/null
+\chapter{Discussion and conclusion}
+
+In this thesis we have seen how the total variation restoration method
+can be extended using an anisotropy tensor, and how this fits into the
+discretization and graph cut framework used in my project work
+\cite{project}. The anisotropy tensor was introduced in hopes of
+reducing the amount of regularization across edges in the image. It was
+constructed based on the structure tensor with three parameters exposed
+for controlling the method.
+
+The continuous functional we initally wanted to minimize was then
+transformed using an anisotropic coarea formula and an anisotropic
+Cauchy--Crofton formula, both described in detail. The functional was
+discretized, and the discretization was shown---under some
+restrictions---to be consistent with the continuous functional.
+
+As in my project work~\cite{project} the discrete functional was
+minimized using maximum flow algorithms to obtain successive minimum
+graph cuts. The description of the push-relabel algorithm was included from
+my project work for completeness, and a description of the
+Boykov--Kolmogorov algorithm was also given, as it is taylored for these
+kinds of graph cut applications.
+
+Effort was put into giving a complete overview of the method, describing
+each part needed to go from the initial continuous problem, to the
+discrete solution. This way, readers can get an understanding of the
+inner workings of the method, and also easily be able to implement it.
+
+Further work is possible in the study of the continuous problem, its
+well-foundedness, and also the anisotropic coarea and Cauchy--Crofton
+formulas, which can be studied on a measure-theoretic foundation.
+\fixme{tensor construction?}
+Regarding the discretization, the choice of neighborhood stencil also
+allows for further discussion, as approximation error can be traded for
+algorithm performance. Opening for non-uniform stencils, where the
+stencil choice depends on the level of detail in the neighborhood is
+also a possiblity.
+
+In the end we looked at how the different parameters affect the
+performance of the algorithm. We will not give any unified conclusion as
+to whether this method is ``better'' or ``worse'' than the regular total
+variation method it is based on, or other methods. In different
+applications the input images have different properties, and one might
+also have different hopes for, and restrictions on the output image. But
+we have seen that the anisotropy can help reduce the contrast loss.
+
+%Because we have spread our attention across all parts of the restoration
+%method, it is also possible to go deeper into the theory behind all of
+%them. The well-foundedness of the continuous formulation and the
+%anisotropic coarea and Cauchy--Crofton formula could have been studied
+%further on a measure-theoretic foundation. Further, in the
+%discretization there are a myriad of choices that could have been
+%discussed further, most notably the construction of the neighborhood
+%stencil, which has a lot to say for the performance, and approximation
+%error of the discrete solution. Further, since we start in the discrete
+%setting with a digital image, and obtain a guaranteed minimizer of the
+%discrete functional, we did not discuss stability and convergence in
+%relation to a theoretical solution to the continuous problem.
+%
+%In the end we looked at how the different parameters affect the
+%performance of the algorithm. We will not give any unified conclusion as
+%to whether this method is ``better'' or ``worse'' than the regular total
+%variation method it is based on, or other methods. In different
+%applications the input images have different properties, and one might
+%also have different hopes for, and restrictions on the output image.
+%Thus I hope a thorough description in addition to an inspection of
+%restoration results has given insight into the good and not-so-good sides
+%of this particular method.
+
\end{subfigure}
\caption[Fidelity energy $F_\lambda^x(1)$ as a function of $\lambda$]{%
Two figures showing how the fidelity energy term $F_\lambda^x(1)$
- in \eqref{eq:total_energy} increases monotonically with $\lambda$.
+ in \eqref{eq:final_discretization} increases monotonically with $\lambda$.
}
\label{fig:norm_evolution}
\chapter{Introduction}
-[image processing]
-
-Image processing forms an important part of our modern computerized
-world. Tasks previously reserved for humans, like detecting edges,
-recognizing textures and inferring shapes and motions can now be
-performed algorithmically. The background of these methods span several
-fields, including phychology and biology for the study of human vision,
-statistics and analysis for the mathematical background, and computer
-science for their implementation and performance analysis.
-
-[image restoration]
+Image processing is becoming an increasingly important part of our
+modern computerized world. Tasks previously only performed by humans,
+like detecting edges, recognizing textures and inferring shapes and
+motions can now be performed algorithmically. The background of these
+methods span several fields, including psychology and biology for the
+study of human vision, statistics and analysis for the mathematical
+background, and computer science for their implementation and
+performance analysis.
Image restoration methods are concerned with trying to remove noise in
-images. There are numerous different ways to approach the problem. For
-background we will discuss some before we go into detail on the total
-variation method.
-
-[total variation]
-
+images. This noise can result from the physical nature of light
+traveling to your sensor, dust on your lens, or many other sources.
+Therefore
+numerous different approaches exist, each having their own
+strengths and weaknesses. Before going into
+detail on the main focus of this thesis---the anisotropic total
+variation method---we will look into some other popular methods.
+
+In my project work \cite{project}, I described a total variation based
+image restoration method, using a graph cut framework for the numerical
+solution.
As the name suggests, total variation is a measure of how much variation
-there is in an image, in total. Thus these methods are often trying to
-reduce the total variation while still staying close to the original
-image, in some sense. The main strength of the total variation method is
-its ability to conserve edges, as it does not favor smooth gradients
+there is in an image. Thus these methods are often trying to
+reduce the total variation while still staying ``close'' to the original
+image, in some sense. The main strength of the total variation
+method is
+its ability to recover edges. The total variation does not favor smooth gradients
over edges like many other methods. However, it may introduce edges from
-noise, and it may also reduce the total contrast in the image.
-
-mention project
-
-[thesis walkthrough]
-
-In this thesis we introduce an anisotropy tensor into the total
-variation norm. This means making the norm directionally dependant, such
-that we can control the weight of the variation based on the position,
-and also the direction. The continuous problem is transformed through an
+noise, and it may also reduce the contrast of the image.
+
+In this thesis we extend the total variation method by introducing an anisotropy
+tensor into the total variation norm. This makes the norm
+directionally dependant, such that we can control the regularization
+applied based on position and direction. The main idea is then to reduce
+the regularization across edges in the image.
+The continuous problem is transformed through an
anisotropic coarea formula and an anisotropic Cauchy--Crofton formula to
-facilitate the discretization. This is one of the main parts of this
-thesis.
+facilitate the discretization.
-The continuous formulation, now consisting of several integrals, is then
-discretized into a series of sums, over the pixels in an image. We make
+The integrals of the continuous formulation are discretized into sums,
+and we make
sure that the discrete formulation is consistent with the continuous
one.
-
A provable optimal solution to the discrete problem is then found using
a graph cut algorithm. For each level in the image, a graph is
-constructed, and a minimum cut is found, and the restored image can then
+constructed, and a minimum cut is found. The restored image can then
be extracted from all these partial solutions.
-For finding these minimum cuts, two algorithms are presented. The
+For finding the minimum cuts, two algorithms are presented. The
push-relabel algorithm is considered to be the fastest and most
versatile for general graphs, while the Boykov--Kolmogorov algorithm is
specially taylored for the type of graphs we find in these kinds of
imaging applications.
-Further we have some results, and then that's it.
+In the end we inspect restoration results to see how the method
+performs. The construction of the tensor exposes three parameters, two
+controlling the scale of the structures it should be sensitive to, and
+one controlling the amount of anisotropy. We experiment with these
+parameters to show that they do what we expect them to do. The method is
+also compared with regular the total variation method to see that some
+improvements are achieved. We also explore and look at the different
+artifacts caused by approximations in the discretization, and how they
+affect the restoration.
%In everyday life cameras are used to capture a moment and save it for
%eternity, but imaging technology technology can be used in many other
+ \beta \sum_{\lambda = 0}^{L-2} \sum_{(x, y)} F^{x,y}(u^\lambda_x,
u^\lambda_y)
=: \sum_{\lambda=0}^{L-2} F_\lambda(u^\lambda)
- \label{eq:total_energy}
+ \label{eq:old_total_energy}
\end{equation}
where the sum over $(x, y)$ is over all pixel pairs $(x, y)$ in a neighbor
relation, i.e.\ where the pixels are ``close'' to each other.
\input{discrete}
\input{maxflow}
\input{results}
+\input{conclusion}
\cleardoublepage
~
\begin{subfigure}[t]{0.30\textwidth}
\centering
- \includegraphics[width=\textwidth]{fig/factory/lines/r_p2_n8_b10000_g100_r4_s2_color.png}
+ \begin{overpic}[width=\textwidth]{fig/factory/lines/r_p2_n8_b10000_g100_r4_s2_color.png}
+ \put(80,0){\includegraphics[scale=0.05]{fig/wheel.png}}
+ \end{overpic}
+ %\includegraphics[width=\textwidth]{fig/factory/lines/r_p2_n8_b10000_g100_r4_s2_color.png}
\caption{%
Restored with relatively high noise scale of $\sigma = 2$.
}
~
\begin{subfigure}[t]{0.30\textwidth}
\centering
- \includegraphics[width=\textwidth]{fig/factory/lines/r_p2_n8_b10000_g100_r15_s0.2_color.png}
+ \begin{overpic}[width=\textwidth]{fig/factory/lines/r_p2_n8_b10000_g100_r15_s0.2_color.png}
+ \put(80,0){\includegraphics[scale=0.05]{fig/wheel.png}}
+ \end{overpic}
+ %\includegraphics[width=\textwidth]{fig/factory/lines/r_p2_n8_b10000_g100_r15_s0.2_color.png}
\caption{%
Restored with low noise scale $\sigma = 0.2$, but large integration
scale $\rho = 15$.
~
\begin{subfigure}[t]{0.30\textwidth}
\centering
- \includegraphics[width=\textwidth]{fig/factory/finger/r_p2_n32_b15000_g150_r10_s3_color.png}
+ \begin{overpic}[width=\textwidth]{fig/factory/finger/r_p2_n32_b15000_g150_r10_s3_color.png}
+ \put(80,80){\includegraphics[scale=0.05]{fig/wheel.png}}
+ \end{overpic}
+ %\includegraphics[width=\textwidth]{fig/factory/finger/r_p2_n32_b15000_g150_r10_s3_color.png}
\caption{%
$\rho = 10$
}
~
\begin{subfigure}[t]{0.30\textwidth}
\centering
- \includegraphics[width=\textwidth]{fig/factory/finger/r_p2_n32_b15000_g150_r20_s3_color.png}
+ \begin{overpic}[width=\textwidth]{fig/factory/finger/r_p2_n32_b15000_g150_r20_s3_color.png}
+ \put(80,80){\includegraphics[scale=0.05]{fig/wheel.png}}
+ \end{overpic}
+ %\includegraphics[width=\textwidth]{fig/factory/finger/r_p2_n32_b15000_g150_r20_s3_color.png}
\caption{%
$\rho = 20$
}
\end{subfigure}
\caption[Anisotropy tensor visualized for a fingerprint test image]{%
A noisy fingerprint with the anisotropy tensor visualized for
- different integration scales $\rho$.
+ different integration scales $\rho$. Parameters:
+ $\abs{\mathcal{N}} = 32$, $\sigma = 3$, $\beta = 15000$, $\omega = 150$.
}
\label{fig:finger_tensor}
\end{figure}
table shows how our discrete Cauchy--Crofton formula approximates the
circumference of circles of different radii. Note that the circumference
approximated is that of an actual continuous circle $u : \mathbb{R}^2
-\to \{0, 1\}$ and not a discrete representation.
+\to \{0, 1\}$ and not a discrete representation. We see that the
+perimeter of the smalles circle is grossly underestimated by the larger
+neighborhoods. Thus the perimeter of one-pixel noise will be
+underestimated, and in turn the contribution to the total variation by
+one-pixel noise will be smaller for large neighborhoods.
\section{Restoration}
\centering
\includegraphics[width=\textwidth]{fig/factory/finger/r_p2_n32_b15000_g150_r10_s3.png}
\caption{%
- Anisotropic TV ($\gamma = 150$), $\beta=15000$,
+ Anisotropic TV ($\omega = 150$), $\beta=15000$,
$\norm{u - f}_{L^2} = 15272$.
}
\label{fig:finger_contrast_atv1}
\centering
\includegraphics[width=\textwidth]{fig/factory/finger/r_p2_n32_b20970_g150_r10_s3.png}
\caption{%
- Anisotropic TV ($\gamma = 150$), $\beta=20970$,
+ Anisotropic TV ($\omega = 150$), $\beta=20970$,
$\norm{u - f}_{L^2} = 17022$.
}
\label{fig:finger_contrast_atv2}
% \label{fig:lena_method_noise}
%\end{figure}
-\chapter{Discussion and conclusion}
-
-In this thesis we have seen how the total variation restoration method
-can be extended using an anisotropy tensor, and how this fits into the
-discretization and graph cut framework used in my project work
-\cite{project}. The anisotropy tensor was introduced in hopes of
-reducing the amount of regularization across edges in the image. It was
-constructed based on the structure tensor with three parameters exposed
-for controlling the method.
-
-The continuous functional we initally wanted to minimize was then
-transformed using an anisotropic coarea formula and an anisotropic
-Cauchy--Crofton formula, both described in detail. The functional was
-discretized, and the discretization was shown---under some
-restrictions---to be consistent with the continuous functional.
-
-As in my project work \cite{project} the discrete functional was
-minimized using maximum flow algorithms to obtain successive minimum
-graph cuts. The description of the push-relabel algorithm was included from
-my project work for completeness, and a description of the
-Boykov--Kolmogorov algorithm was also given, as it is taylored for these
-kinds of graph cut applications.
-
-Because we have spread our attention across all parts of the restoration
-method, it is also possible to go deeper into the theory behind all of
-them. The well-foundedness of the continuous formulation and the
-anisotropic coarea and Cauchy--Crofton formula could have been studied
-further on a measure-theoretic foundation. Further, in the
-discretization there are a myriad of choices that could have been
-discussed further, most notably the construction of the neighborhood
-stencil, which has a lot to say for the performance, and approximation
-error of the discrete solution. Further, since we start in the discrete
-setting with a digital image, and obtain a guaranteed minimizer of the
-discrete functional, we did not discuss stability and convergence in
-relation to a theoretical solution to the continuous problem.
-
-In the end we looked at how the different parameters affect the
-performance of the algorithm. We will not give any unified conclusion as
-to whether this method is ``better'' or ``worse'' than the regular total
-variation method it is based on, or other methods. In different
-applications the input images have different properties, and one might
-also have different hopes for, and restrictions on the output image.
-Thus I hope a thorough description in addition to an inspection of
-restoration results has given insight into the good and not-so-good sides
-of this particular method.
-
projects / master / commitdiff