--- /dev/null
+\centering
+\begin{subfigure}[t]{0.4\textwidth}
+ \centering
+ \begin{tikzpicture}[scale=1.4]
+ \fill[color=black!15!white] plot[smooth] coordinates {(0,0) (1,1) (3,2) (3,2)
+ (3,2) (3,0)};
+ \begin{scope}[shift={(1,1)},rotate=37]
+ \draw[color=red,thick] (0, 0) ellipse (0.5cm and 1cm);
+ \draw[thick,->] (0, 0) --
+ node[font=\footnotesize,shift={(-.1cm,-.2cm)}] {$\lambda_1$} +(0, 1cm);
+ \draw[thick,->] (0, 0) --
+ node[font=\footnotesize,shift={(-.16cm, .22cm)}]
+ {$\lambda_2$} +(0.5cm, 0);
+ \end{scope}
+ \end{tikzpicture}
+ \caption{%
+ The structure tensor $S_\rho$.
+ }
+\end{subfigure}
+~
+\begin{subfigure}[t]{0.4\textwidth}
+ \centering
+ \begin{tikzpicture}[scale=1.4]
+ \fill[color=black!15!white] plot[smooth] coordinates {(0,0) (1,1) (3,2) (3,2)
+ (3,2) (3,0)};
+ \begin{scope}[shift={(1,1)},rotate=127]
+ \draw[color=red,thick] (0, 0) ellipse (0.4cm and 1.2cm);
+ \draw[thick,->] (0, 0) --
+ node[font=\footnotesize,shift={(-0.1cm,0.2cm)}] {$1$} +(0, -1.2cm);
+ \draw[thick,->] (0, 0) --
+ node[font=\footnotesize,shift={(-0.17cm,-0.16cm)}]
+ {$\sigma_1$} +(0.4cm, 0);
+ \end{scope}
+ \end{tikzpicture}
+ \caption{%
+ The anisotropy tensor $A$.
+ }
+\end{subfigure}
+
+\caption{%
+ An edge with the structure and anisotropy tensors visualized as
+ using their eigenvectors and eigenvalues.
+}
+\label{fig:tensor_viz}
``normal'' image to look in the capturing conditions, and also what
kinds of noise we expect to be a part of the image.
-In the next section we will see a small overview of some of the most
-popular methods in image restoration, but in the rest of the report we
-will focus on a total variation based method. One of the strong points
-of these methods are their ability to preserve edges, instead of
-smoothing over them. Different approaches to total variation image
-restoration exist, but we will formulate it as an energy minimization
-problem.
-
-As our input images are digital images, the first step will be to
-discretize the energy function. Next comes an important part of this
-project which is to reformulate the minimization problem as a series of
-minimum cut problems from graph theory. This means we have to carefully
-construct graphs and verify that finding the minimum cuts actually
-yield a global minimizer of the original energy function.
-
-A few different algorithms for finding these minumum cuts are
-considered, but we will mainly focus on the push-relabel algorithm and
-exploit its ability to re-use some of the results across the separate
-subproblems.
-
-Towards the end we will look at some image restoration results showing
-how the method performs for different kinds of noise, and for different
-input parameters.
+In the next chapter we will see a short overview of some of the most
+popular methods in image restoration, especially those related to the
+anisotropic total variation based method that will be the focus of the
+rest of the thesis.
+
+In my project work \cite{project}, I described how total variation
+regularization can be performed on a grayscale image using a graph cut
+approach. In this thesis, this method will be expanded upon by
+introducing a more flexible \emph{anisotropic} total variation. The goal
+is to improve how the method behaves around edges and detailed parts of
+the image.
+
+In order to get a good understanding of the method, we will try to
+describe and look into all the different parts needed to make the method work,
+instead of focusing particularly on one piece of the puzzle. The
+continuous formulation and the theorems relating to it are thoroughly
+described, as they are central to how we arrive at our discrete minimizer.
+Next we look at the discretization of the functional, and its different
+approximation errors. The discrete minimizer is found using a graph cut
+approach as in my project work, and the description is similar except
+for some minor changes and a new section on the Boykov--Kolmogorov
+algorithm.
+
+In the end we will through restoration of more or less constructed
+examples look at how the introduced anisotropy changes the method. As
+the anisotropic total variation also introduces several new parameters,
+the result section will focus on looking at how these parameters affect
+the restoration performance.
\chapter{Methods in image restoration}
}
\end{subfigure}
\caption{%
- Funny caption. Parameters.
+ A noisy circle first restored using a uniform tensor, resulting
+ mostly in smoothing in the $x$-direction. Then restored using
+ the tensor described in Section~\ref{sec:anisotropy_tensor}.
}
\label{fig:tensor_experiment}
\end{figure}
\label{fig:lena_process_restored}
\end{subfigure}
\caption{%
- Funny caption. Parameters.
+ The different stages of the restoration algorithm, showing the
+ original image, the smoothed image, the edge detector, two
+ visualizations of the anisotropy tensor, and finally the
+ restored image.
}
\label{fig:lena_process}
\end{figure}
\label{fig:scale_comp_int}
\end{subfigure}
\caption{%
- Funny caption. Parameters.
+ An example constructed to show the effects of the parameters
+ $\sigma$ and $\rho$ in the anisotropy tensor. The noise scale
+ $\sigma$ controls what is considered to be noise, while the
+ integration scale $\rho$ controls the size of the structures
+ considered by the tensor.
}
\label{fig:scale_comp}
\end{figure}
\begin{table}
\caption{%
- The circumference of different circles measured by the
+ The circumference of circles of different radii $r$ measured by the
discretized Cauchy--Crofton formula in \eqref{ref}, using different
neighborhood stencils.
}
\centering
- \begin{tabular}{cccc}
+ \begin{tabular}{ccccc}
\hline
- $\abs{\mathcal{N}}$ & $r = 0.5$ & $r = 5.5$ & $r = 50.5$ \\
+ $\abs{\mathcal{N}}$ & $r = 0.5$ & $r = 1.5$ & $r = 5.5$ & $r = 50.5$ \\
\hline
- 4 & 3.14 & 34.56 & 317.3 \\
- 8 & 2.68 & 33.94 & 317.5 \\
- 16 & 2.08 & 34.59 & 316.8 \\
- 32 & 1.63 & 34.44 & 317.2 \\
- 48 & 1.40 & 34.29 & 317.3 \\
- 72 & 1.21 & 33.95 & 317.2 \\
+ 4 & 3.14 & 9.42 & 34.56 & 317.3 \\
+ 8 & 2.68 & 10.27 & 33.94 & 317.5 \\
+ 16 & 2.08 & 9.97 & 34.59 & 316.8 \\
+ 32 & 1.63 & 9.24 & 34.44 & 317.2 \\
+ 48 & 1.40 & 8.40 & 34.29 & 317.3 \\
+ 72 & 1.21 & 7.45 & 33.95 & 317.2 \\
\hline
- $2\pi r$ & 3.14 & 34.56 & 317.3 \\
+ $2\pi r$ & 3.14 & 9.42 & 34.56 & 317.3 \\
\hline
\end{tabular}
+ \label{tab:circumference}
\end{table}
-As mentioned before, there is another discretization error which relates
-to the length of the edges in the neighborhood. Thus a larger
-neighborhood is not always better, even if it will reduced the artifacts
+As mentioned before, there is a discretization error which relates
+to the length of the edges in the neighborhood. We approximated the
+number of times an edge $e_{ab}$ crosses level set boundary by
+$\abs{u^\lambda_a
+- u^\lambda_b}$, an approximation that becomes worse for long edges. Thus a larger
+neighborhood is not always better, even if it will reduce the artifacts
discussed above. In Figure~\ref{fig:lena_neigh}, a noisy image of Lena
has been restored using two differently sized neighborhood stencils, and
there are obvious differences. For the stencil of size 72, the restored
-image still contains some salt- and pepper-like noise, seemingly very
+image still contains some pixel-sized noise, seemingly very
different from their neighboring pixels. An explanation can be found in
-Figure~\ref{fig:pixel_perimeter}, where we see that the length of a one-pixel curve is
-very underestimated because so many edges cross over it, and thus ignore
-it completely.
+Figure~\ref{fig:pixel_perimeter}. We see that the length of a
+one-pixel curve is underestimated by the Cauchy--Crofton formula because
+many edges cross the curve cross while $\abs{u^\lambda_a - u^\lambda_b}
+= 0$, and thus these edges are ignored completely in our perimeter
+approximation. An additional demonstration that this problem mostly relates
+to small sized noise is shown in Table~\ref{tab:circumference}. The
+table shows how our discrete Cauchy--Crofton formula approximates the
+circumference of circles of different radii. Note that the circumference
+approximated is that of an actual continuous circle $r = x^2 + y^2$ and
+not a discrete representation.
\begin{figure}
\centering
regularization. An example of this can be seen in
Figure~\ref{fig:finger_contrast} where a portion of a fingerprint with
added Gaussian noise has been restored in two different ways. We see
-that when the anisotropy is increased, more of the contrast between dark
+that when the anisotropy $\omega$ is increased, more of the contrast between dark
and light portions of the fingerprint is retained. This can also be seen
in Figure~\ref{fig:contrast_plot} where a one-dimensional slice has been
taken through Figure~\ref{fig:finger_contrast_tv}
-and~\ref{fig:finger_contrast_atv}. Although much of the same structure
-is found for this particular regularization parameter $\beta$, the peaks
-are much higher for the anisotropic total variation.
+and~\ref{fig:finger_contrast_atv}. Please note that this can be seen as
+cheating. The anisotropy tensor, with one eigenvalue equal to 1, and the
+other less or equal to 1, means that when all else being equal, the
+amount of restoration will be reduced.
\begin{figure}
\centering
% \label{fig:lena_method_noise}
%\end{figure}
-\chapter{Conclusion}
-
-Bleep bloop. Bad sides, good sides?
+\chapter{Discussion and conclusion}
+
+In this thesis we have seen how the total variation restoration method
+can be extended using an anisotropy tensor, and how this fits into the
+discretization and graph cut framework used in my project work
+\cite{project}. The anisotropy tensor was introduced in hopes of
+reducing the amount of regularization across edges in the image. It was
+constructed based on the structure tensor with three parameters exposed
+for controlling the method.
+
+The continuous functional we initally wanted to minimize was then
+transformed using an anisotropic coarea formula and an anisotropic
+Cauchy--Crofton formula, both described in detail. The functional was
+discretized, and the discretization was shown---under some
+restrictions---to be consistent with the continuous functional.
+
+As in my project work \cite{project} the discrete functional was
+minimized using maximum flow algorithms to obtain successive minimum
+graph cuts. The description of the push-relabel algorithm was included from
+my project work for completeness, and a description of the
+Boykov--Kolmogorov algorithm was also given, as it is taylored for these
+kinds of graph cut applications.
+
+Because we have spread our attention across all parts of the restoration
+method, it is also possible to go deeper into the theory behind all of
+them. The well-foundedness of the continuous formulation and the
+anisotropic coarea and Cauchy--Crofton formula could have been studied
+further on a measure-theoretic foundation. Further, in the
+discretization there are a myriad of choices that could have been
+discussed further, most notably the construction of the neighborhood
+stencil, which has a lot to say for the performance, and approximation
+error of the discrete solution. Further, since we start in the discrete
+setting with a digital image, and obtain a guaranteed minimizer of the
+discrete functional, we did not discuss stability and convergence in
+relation to a theoretical solution to the continuous problem.
+
+In the end we looked at how the different parameters affect the
+performance of the algorithm. We will not give any unified conclusion as
+to whether this method is ``better'' or ``worse'' than the regular total
+variation method it is based on, or other methods. In different
+applications the input images have different properties, and one might
+also have different hopes for, and restrictions on the output image.
+Thus I hope a thorough description in addition to an inspection of
+restoration results can give insight into the good and not-so-good sides
+of this particular method.
\label{eq:sigma_construction}
\end{equation}
Thus the eigenvectors of $A(x)$ and $S_\rho(x)$ are equal, while the
-eigenvalues are different. In smooth areas, $\sigma_1 \approx 1$ and
+eigenvalues are different. A visualization of the two tensors can be
+seen in Figure~\ref{fig:tensor_viz} where the two tensors are shown at
+an edge in the image.
+
+In smooth areas, $\sigma_1 \approx 1$ and
$A(x)$ will be close to the identity matrix, while $A(x)$ will reduce
the effect of $\nabla u$ across edges.
-The parameter $\omega$ controls the amount of anisotropy in the method.
+The parameter $\omega$ controls the amount of anisotropy in the method,
+such that if it is very large we are left with the identity matrix.
Around corners $A(x)$ will be close to the identity matrix, which gives
regularization similar to smooth areas. This is one possible down-side
from Weickert. Discuss that this is not optimal in corners.
}
-\fixme{%
- This should be visualized, with a figure showing the length and
- direction of the eigenvalues in the area around an edge.
-}
+\begin{figure}
+ \input{fig/tensor_viz}
+\end{figure}
\fixme{%
The numerical problems should be discussed somewhere but maybe not
\fixme{Sequential coercivity?}
+Coercivity relates to how the functional behaves when the norm of the
+image $u$ tends to infinity.
It is obvious from the fidelity term that for some fixed $f \in
-L^2(\Omega)$ if $\lnorm{u}_{L^2} \to \infty$ then $F(u) \to \infty$.
+L^2(\Omega)$, if $\lnorm{u}_{L^2} \to \infty$ then $F(u) \to \infty$.
This tells us something about the locality of potential minimizers.
\begin{figure}
projects / master / commitdiff