From: Bjørn Rustad Date: Wed, 21 Jan 2015 16:40:41 +0000 (+0100) Subject: BEEP X-Git-Url: http://git.rustad.me/?a=commitdiff_plain;h=64a0d9a78066eb8e2c669f432537b4df0c6eea49;p=master BEEP --- diff --git a/Makefile b/Makefile index bf77339..446834e 100644 --- a/Makefile +++ b/Makefile @@ -5,3 +5,6 @@ all: makeindex main.nlo -s nomencl.ist -o main.nls xelatex -shell-escape main cp main.pdf ~/Dropbox/master/ + +quick: + xelatex -shell-escape main diff --git a/fig/tensor_viz.tex b/fig/tensor_viz.tex new file mode 100644 index 0000000..57d7ae2 --- /dev/null +++ b/fig/tensor_viz.tex @@ -0,0 +1,44 @@ +\centering +\begin{subfigure}[t]{0.4\textwidth} + \centering + \begin{tikzpicture}[scale=1.4] + \fill[color=black!15!white] plot[smooth] coordinates {(0,0) (1,1) (3,2) (3,2) + (3,2) (3,0)}; + \begin{scope}[shift={(1,1)},rotate=37] + \draw[color=red,thick] (0, 0) ellipse (0.5cm and 1cm); + \draw[thick,->] (0, 0) -- + node[font=\footnotesize,shift={(-.1cm,-.2cm)}] {$\lambda_1$} +(0, 1cm); + \draw[thick,->] (0, 0) -- + node[font=\footnotesize,shift={(-.16cm, .22cm)}] + {$\lambda_2$} +(0.5cm, 0); + \end{scope} + \end{tikzpicture} + \caption{% + The structure tensor $S_\rho$. + } +\end{subfigure} +~ +\begin{subfigure}[t]{0.4\textwidth} + \centering + \begin{tikzpicture}[scale=1.4] + \fill[color=black!15!white] plot[smooth] coordinates {(0,0) (1,1) (3,2) (3,2) + (3,2) (3,0)}; + \begin{scope}[shift={(1,1)},rotate=127] + \draw[color=red,thick] (0, 0) ellipse (0.4cm and 1.2cm); + \draw[thick,->] (0, 0) -- + node[font=\footnotesize,shift={(-0.1cm,0.2cm)}] {$1$} +(0, -1.2cm); + \draw[thick,->] (0, 0) -- + node[font=\footnotesize,shift={(-0.17cm,-0.16cm)}] + {$\sigma_1$} +(0.4cm, 0); + \end{scope} + \end{tikzpicture} + \caption{% + The anisotropy tensor $A$. + } +\end{subfigure} + +\caption{% + An edge with the structure and anisotropy tensors visualized as + using their eigenvectors and eigenvalues. +} +\label{fig:tensor_viz} diff --git a/introduction.tex b/introduction.tex index 9e51f9d..f7c2ac5 100644 --- a/introduction.tex +++ b/introduction.tex @@ -32,29 +32,34 @@ image. These methods often take into account how we would expect a ``normal'' image to look in the capturing conditions, and also what kinds of noise we expect to be a part of the image. -In the next section we will see a small overview of some of the most -popular methods in image restoration, but in the rest of the report we -will focus on a total variation based method. One of the strong points -of these methods are their ability to preserve edges, instead of -smoothing over them. Different approaches to total variation image -restoration exist, but we will formulate it as an energy minimization -problem. - -As our input images are digital images, the first step will be to -discretize the energy function. Next comes an important part of this -project which is to reformulate the minimization problem as a series of -minimum cut problems from graph theory. This means we have to carefully -construct graphs and verify that finding the minimum cuts actually -yield a global minimizer of the original energy function. - -A few different algorithms for finding these minumum cuts are -considered, but we will mainly focus on the push-relabel algorithm and -exploit its ability to re-use some of the results across the separate -subproblems. - -Towards the end we will look at some image restoration results showing -how the method performs for different kinds of noise, and for different -input parameters. +In the next chapter we will see a short overview of some of the most +popular methods in image restoration, especially those related to the +anisotropic total variation based method that will be the focus of the +rest of the thesis. + +In my project work \cite{project}, I described how total variation +regularization can be performed on a grayscale image using a graph cut +approach. In this thesis, this method will be expanded upon by +introducing a more flexible \emph{anisotropic} total variation. The goal +is to improve how the method behaves around edges and detailed parts of +the image. + +In order to get a good understanding of the method, we will try to +describe and look into all the different parts needed to make the method work, +instead of focusing particularly on one piece of the puzzle. The +continuous formulation and the theorems relating to it are thoroughly +described, as they are central to how we arrive at our discrete minimizer. +Next we look at the discretization of the functional, and its different +approximation errors. The discrete minimizer is found using a graph cut +approach as in my project work, and the description is similar except +for some minor changes and a new section on the Boykov--Kolmogorov +algorithm. + +In the end we will through restoration of more or less constructed +examples look at how the introduced anisotropy changes the method. As +the anisotropic total variation also introduces several new parameters, +the result section will focus on looking at how these parameters affect +the restoration performance. \chapter{Methods in image restoration} diff --git a/main.tex b/main.tex index 67be508..4d911f9 100644 --- a/main.tex +++ b/main.tex @@ -10,6 +10,7 @@ %\setmainfont[Ligatures=TeX]{TG Termes Math} %\setmainfont[Mapping=tex-text,Ligatures=TeX]{TeX Gyre Pagella} %\setmathfont[Mapping=tex-text,Ligatures=TeX]{TeX Gyre Pagella Math} +\usepackage{lmodern} \usepackage{amsthm} \usepackage{graphicx} \usepackage[percent]{overpic} @@ -99,15 +100,29 @@ \cleardoublepage -\abstract{% - BLEEP BLOOP. -} +%\abstract{% +% BLEEP BLOOP. +%} + +\section*{Abstract} + +Lelelel. + +\cleardoublepage + +%\abstract{% +% BLEEP BLEEP NORSK. +%} + +\section*{Sammendrag} + +Lololol. \cleardoublepage -\abstract{% - BLEEP BLEEP NORSK. -} +\section*{Preface} + +Bleep bloop. Acknowledements? % Romerske tall pÃ¥ alt før selve rapporten starter er pent. \pagenumbering{roman} diff --git a/results.tex b/results.tex index 545cdd7..f3b43fe 100644 --- a/results.tex +++ b/results.tex @@ -35,7 +35,9 @@ see how it affects the performance of the restoration algorithm. } \end{subfigure} \caption{% - Funny caption. Parameters. + A noisy circle first restored using a uniform tensor, resulting + mostly in smoothing in the $x$-direction. Then restored using + the tensor described in Section~\ref{sec:anisotropy_tensor}. } \label{fig:tensor_experiment} \end{figure} @@ -117,7 +119,10 @@ the $y$-direction. \label{fig:lena_process_restored} \end{subfigure} \caption{% - Funny caption. Parameters. + The different stages of the restoration algorithm, showing the + original image, the smoothed image, the edge detector, two + visualizations of the anisotropy tensor, and finally the + restored image. } \label{fig:lena_process} \end{figure} @@ -202,7 +207,11 @@ image, and we see that it has been heavily regularized. \label{fig:scale_comp_int} \end{subfigure} \caption{% - Funny caption. Parameters. + An example constructed to show the effects of the parameters + $\sigma$ and $\rho$ in the anisotropy tensor. The noise scale + $\sigma$ controls what is considered to be noise, while the + integration scale $\rho$ controls the size of the structures + considered by the tensor. } \label{fig:scale_comp} \end{figure} @@ -413,38 +422,49 @@ some. \begin{table} \caption{% - The circumference of different circles measured by the + The circumference of circles of different radii $r$ measured by the discretized Cauchy--Crofton formula in \eqref{ref}, using different neighborhood stencils. } \centering - \begin{tabular}{cccc} + \begin{tabular}{ccccc} \hline - $\abs{\mathcal{N}}$ & $r = 0.5$ & $r = 5.5$ & $r = 50.5$ \\ + $\abs{\mathcal{N}}$ & $r = 0.5$ & $r = 1.5$ & $r = 5.5$ & $r = 50.5$ \\ \hline - 4 & 3.14 & 34.56 & 317.3 \\ - 8 & 2.68 & 33.94 & 317.5 \\ - 16 & 2.08 & 34.59 & 316.8 \\ - 32 & 1.63 & 34.44 & 317.2 \\ - 48 & 1.40 & 34.29 & 317.3 \\ - 72 & 1.21 & 33.95 & 317.2 \\ + 4 & 3.14 & 9.42 & 34.56 & 317.3 \\ + 8 & 2.68 & 10.27 & 33.94 & 317.5 \\ + 16 & 2.08 & 9.97 & 34.59 & 316.8 \\ + 32 & 1.63 & 9.24 & 34.44 & 317.2 \\ + 48 & 1.40 & 8.40 & 34.29 & 317.3 \\ + 72 & 1.21 & 7.45 & 33.95 & 317.2 \\ \hline - $2\pi r$ & 3.14 & 34.56 & 317.3 \\ + $2\pi r$ & 3.14 & 9.42 & 34.56 & 317.3 \\ \hline \end{tabular} + \label{tab:circumference} \end{table} -As mentioned before, there is another discretization error which relates -to the length of the edges in the neighborhood. Thus a larger -neighborhood is not always better, even if it will reduced the artifacts +As mentioned before, there is a discretization error which relates +to the length of the edges in the neighborhood. We approximated the +number of times an edge $e_{ab}$ crosses level set boundary by +$\abs{u^\lambda_a +- u^\lambda_b}$, an approximation that becomes worse for long edges. Thus a larger +neighborhood is not always better, even if it will reduce the artifacts discussed above. In Figure~\ref{fig:lena_neigh}, a noisy image of Lena has been restored using two differently sized neighborhood stencils, and there are obvious differences. For the stencil of size 72, the restored -image still contains some salt- and pepper-like noise, seemingly very +image still contains some pixel-sized noise, seemingly very different from their neighboring pixels. An explanation can be found in -Figure~\ref{fig:pixel_perimeter}, where we see that the length of a one-pixel curve is -very underestimated because so many edges cross over it, and thus ignore -it completely. +Figure~\ref{fig:pixel_perimeter}. We see that the length of a +one-pixel curve is underestimated by the Cauchy--Crofton formula because +many edges cross the curve cross while $\abs{u^\lambda_a - u^\lambda_b} += 0$, and thus these edges are ignored completely in our perimeter +approximation. An additional demonstration that this problem mostly relates +to small sized noise is shown in Table~\ref{tab:circumference}. The +table shows how our discrete Cauchy--Crofton formula approximates the +circumference of circles of different radii. Note that the circumference +approximated is that of an actual continuous circle $r = x^2 + y^2$ and +not a discrete representation. \begin{figure} \centering @@ -485,13 +505,14 @@ the contrast loss one can encounter with regular total variation regularization. An example of this can be seen in Figure~\ref{fig:finger_contrast} where a portion of a fingerprint with added Gaussian noise has been restored in two different ways. We see -that when the anisotropy is increased, more of the contrast between dark +that when the anisotropy $\omega$ is increased, more of the contrast between dark and light portions of the fingerprint is retained. This can also be seen in Figure~\ref{fig:contrast_plot} where a one-dimensional slice has been taken through Figure~\ref{fig:finger_contrast_tv} -and~\ref{fig:finger_contrast_atv}. Although much of the same structure -is found for this particular regularization parameter $\beta$, the peaks -are much higher for the anisotropic total variation. +and~\ref{fig:finger_contrast_atv}. Please note that this can be seen as +cheating. The anisotropy tensor, with one eigenvalue equal to 1, and the +other less or equal to 1, means that when all else being equal, the +amount of restoration will be reduced. \begin{figure} \centering @@ -567,7 +588,49 @@ the image. % \label{fig:lena_method_noise} %\end{figure} -\chapter{Conclusion} - -Bleep bloop. Bad sides, good sides? +\chapter{Discussion and conclusion} + +In this thesis we have seen how the total variation restoration method +can be extended using an anisotropy tensor, and how this fits into the +discretization and graph cut framework used in my project work +\cite{project}. The anisotropy tensor was introduced in hopes of +reducing the amount of regularization across edges in the image. It was +constructed based on the structure tensor with three parameters exposed +for controlling the method. + +The continuous functional we initally wanted to minimize was then +transformed using an anisotropic coarea formula and an anisotropic +Cauchy--Crofton formula, both described in detail. The functional was +discretized, and the discretization was shown---under some +restrictions---to be consistent with the continuous functional. + +As in my project work \cite{project} the discrete functional was +minimized using maximum flow algorithms to obtain successive minimum +graph cuts. The description of the push-relabel algorithm was included from +my project work for completeness, and a description of the +Boykov--Kolmogorov algorithm was also given, as it is taylored for these +kinds of graph cut applications. + +Because we have spread our attention across all parts of the restoration +method, it is also possible to go deeper into the theory behind all of +them. The well-foundedness of the continuous formulation and the +anisotropic coarea and Cauchy--Crofton formula could have been studied +further on a measure-theoretic foundation. Further, in the +discretization there are a myriad of choices that could have been +discussed further, most notably the construction of the neighborhood +stencil, which has a lot to say for the performance, and approximation +error of the discrete solution. Further, since we start in the discrete +setting with a digital image, and obtain a guaranteed minimizer of the +discrete functional, we did not discuss stability and convergence in +relation to a theoretical solution to the continuous problem. + +In the end we looked at how the different parameters affect the +performance of the algorithm. We will not give any unified conclusion as +to whether this method is ``better'' or ``worse'' than the regular total +variation method it is based on, or other methods. In different +applications the input images have different properties, and one might +also have different hopes for, and restrictions on the output image. +Thus I hope a thorough description in addition to an inspection of +restoration results can give insight into the good and not-so-good sides +of this particular method. diff --git a/theory.tex b/theory.tex index 177996b..1b20230 100644 --- a/theory.tex +++ b/theory.tex @@ -224,11 +224,16 @@ and for $\sigma_1$ and $\sigma_2$ we choose \label{eq:sigma_construction} \end{equation} Thus the eigenvectors of $A(x)$ and $S_\rho(x)$ are equal, while the -eigenvalues are different. In smooth areas, $\sigma_1 \approx 1$ and +eigenvalues are different. A visualization of the two tensors can be +seen in Figure~\ref{fig:tensor_viz} where the two tensors are shown at +an edge in the image. + +In smooth areas, $\sigma_1 \approx 1$ and $A(x)$ will be close to the identity matrix, while $A(x)$ will reduce the effect of $\nabla u$ across edges. -The parameter $\omega$ controls the amount of anisotropy in the method. +The parameter $\omega$ controls the amount of anisotropy in the method, +such that if it is very large we are left with the identity matrix. Around corners $A(x)$ will be close to the identity matrix, which gives regularization similar to smooth areas. This is one possible down-side @@ -244,10 +249,9 @@ suffice for $U(x)$. from Weickert. Discuss that this is not optimal in corners. } -\fixme{% - This should be visualized, with a figure showing the length and - direction of the eigenvalues in the area around an edge. -} +\begin{figure} + \input{fig/tensor_viz} +\end{figure} \fixme{% The numerical problems should be discussed somewhere but maybe not @@ -309,8 +313,10 @@ given the existence of a minimizer, implies uniqueness. \fixme{Sequential coercivity?} +Coercivity relates to how the functional behaves when the norm of the +image $u$ tends to infinity. It is obvious from the fidelity term that for some fixed $f \in -L^2(\Omega)$ if $\lnorm{u}_{L^2} \to \infty$ then $F(u) \to \infty$. +L^2(\Omega)$, if $\lnorm{u}_{L^2} \to \infty$ then $F(u) \to \infty$. This tells us something about the locality of potential minimizers. \begin{figure}