From: Bjørn Rustad Date: Sun, 1 Feb 2015 17:41:05 +0000 (+0100) Subject: BLEEP X-Git-Url: http://git.rustad.me/?a=commitdiff_plain;h=ed1590da532945ea40f057f1262ae3fae94ee9ce;p=master BLEEP --- diff --git a/conclusion.tex b/conclusion.tex new file mode 100644 index 0000000..228b169 --- /dev/null +++ b/conclusion.tex @@ -0,0 +1,69 @@ +\chapter{Discussion and conclusion} + +In this thesis we have seen how the total variation restoration method +can be extended using an anisotropy tensor, and how this fits into the +discretization and graph cut framework used in my project work +\cite{project}. The anisotropy tensor was introduced in hopes of +reducing the amount of regularization across edges in the image. It was +constructed based on the structure tensor with three parameters exposed +for controlling the method. + +The continuous functional we initally wanted to minimize was then +transformed using an anisotropic coarea formula and an anisotropic +Cauchy--Crofton formula, both described in detail. The functional was +discretized, and the discretization was shown---under some +restrictions---to be consistent with the continuous functional. + +As in my project work~\cite{project} the discrete functional was +minimized using maximum flow algorithms to obtain successive minimum +graph cuts. The description of the push-relabel algorithm was included from +my project work for completeness, and a description of the +Boykov--Kolmogorov algorithm was also given, as it is taylored for these +kinds of graph cut applications. + +Effort was put into giving a complete overview of the method, describing +each part needed to go from the initial continuous problem, to the +discrete solution. This way, readers can get an understanding of the +inner workings of the method, and also easily be able to implement it. + +Further work is possible in the study of the continuous problem, its +well-foundedness, and also the anisotropic coarea and Cauchy--Crofton +formulas, which can be studied on a measure-theoretic foundation. +\fixme{tensor construction?} +Regarding the discretization, the choice of neighborhood stencil also +allows for further discussion, as approximation error can be traded for +algorithm performance. Opening for non-uniform stencils, where the +stencil choice depends on the level of detail in the neighborhood is +also a possiblity. + +In the end we looked at how the different parameters affect the +performance of the algorithm. We will not give any unified conclusion as +to whether this method is ``better'' or ``worse'' than the regular total +variation method it is based on, or other methods. In different +applications the input images have different properties, and one might +also have different hopes for, and restrictions on the output image. But +we have seen that the anisotropy can help reduce the contrast loss. + +%Because we have spread our attention across all parts of the restoration +%method, it is also possible to go deeper into the theory behind all of +%them. The well-foundedness of the continuous formulation and the +%anisotropic coarea and Cauchy--Crofton formula could have been studied +%further on a measure-theoretic foundation. Further, in the +%discretization there are a myriad of choices that could have been +%discussed further, most notably the construction of the neighborhood +%stencil, which has a lot to say for the performance, and approximation +%error of the discrete solution. Further, since we start in the discrete +%setting with a digital image, and obtain a guaranteed minimizer of the +%discrete functional, we did not discuss stability and convergence in +%relation to a theoretical solution to the continuous problem. +% +%In the end we looked at how the different parameters affect the +%performance of the algorithm. We will not give any unified conclusion as +%to whether this method is ``better'' or ``worse'' than the regular total +%variation method it is based on, or other methods. In different +%applications the input images have different properties, and one might +%also have different hopes for, and restrictions on the output image. +%Thus I hope a thorough description in addition to an inspection of +%restoration results has given insight into the good and not-so-good sides +%of this particular method. + diff --git a/fig/factory/lena_eye/n_q30.pgm b/fig/factory/lena_eye/n_q30.pgm index 2178676..334a132 100644 Binary files a/fig/factory/lena_eye/n_q30.pgm and b/fig/factory/lena_eye/n_q30.pgm differ diff --git a/fig/norm_evolution.tex b/fig/norm_evolution.tex index 7a20014..ed85765 100644 --- a/fig/norm_evolution.tex +++ b/fig/norm_evolution.tex @@ -50,6 +50,6 @@ \end{subfigure} \caption[Fidelity energy $F_\lambda^x(1)$ as a function of $\lambda$]{% Two figures showing how the fidelity energy term $F_\lambda^x(1)$ - in \eqref{eq:total_energy} increases monotonically with $\lambda$. + in \eqref{eq:final_discretization} increases monotonically with $\lambda$. } \label{fig:norm_evolution} diff --git a/introduction.tex b/introduction.tex index 6ce490b..e020ab8 100644 --- a/introduction.tex +++ b/introduction.tex @@ -1,61 +1,68 @@ \chapter{Introduction} -[image processing] - -Image processing forms an important part of our modern computerized -world. Tasks previously reserved for humans, like detecting edges, -recognizing textures and inferring shapes and motions can now be -performed algorithmically. The background of these methods span several -fields, including phychology and biology for the study of human vision, -statistics and analysis for the mathematical background, and computer -science for their implementation and performance analysis. - -[image restoration] +Image processing is becoming an increasingly important part of our +modern computerized world. Tasks previously only performed by humans, +like detecting edges, recognizing textures and inferring shapes and +motions can now be performed algorithmically. The background of these +methods span several fields, including psychology and biology for the +study of human vision, statistics and analysis for the mathematical +background, and computer science for their implementation and +performance analysis. Image restoration methods are concerned with trying to remove noise in -images. There are numerous different ways to approach the problem. For -background we will discuss some before we go into detail on the total -variation method. - -[total variation] - +images. This noise can result from the physical nature of light +traveling to your sensor, dust on your lens, or many other sources. +Therefore +numerous different approaches exist, each having their own +strengths and weaknesses. Before going into +detail on the main focus of this thesis---the anisotropic total +variation method---we will look into some other popular methods. + +In my project work \cite{project}, I described a total variation based +image restoration method, using a graph cut framework for the numerical +solution. As the name suggests, total variation is a measure of how much variation -there is in an image, in total. Thus these methods are often trying to -reduce the total variation while still staying close to the original -image, in some sense. The main strength of the total variation method is -its ability to conserve edges, as it does not favor smooth gradients +there is in an image. Thus these methods are often trying to +reduce the total variation while still staying ``close'' to the original +image, in some sense. The main strength of the total variation +method is +its ability to recover edges. The total variation does not favor smooth gradients over edges like many other methods. However, it may introduce edges from -noise, and it may also reduce the total contrast in the image. - -mention project - -[thesis walkthrough] - -In this thesis we introduce an anisotropy tensor into the total -variation norm. This means making the norm directionally dependant, such -that we can control the weight of the variation based on the position, -and also the direction. The continuous problem is transformed through an +noise, and it may also reduce the contrast of the image. + +In this thesis we extend the total variation method by introducing an anisotropy +tensor into the total variation norm. This makes the norm +directionally dependant, such that we can control the regularization +applied based on position and direction. The main idea is then to reduce +the regularization across edges in the image. +The continuous problem is transformed through an anisotropic coarea formula and an anisotropic Cauchy--Crofton formula to -facilitate the discretization. This is one of the main parts of this -thesis. +facilitate the discretization. -The continuous formulation, now consisting of several integrals, is then -discretized into a series of sums, over the pixels in an image. We make +The integrals of the continuous formulation are discretized into sums, +and we make sure that the discrete formulation is consistent with the continuous one. - A provable optimal solution to the discrete problem is then found using a graph cut algorithm. For each level in the image, a graph is -constructed, and a minimum cut is found, and the restored image can then +constructed, and a minimum cut is found. The restored image can then be extracted from all these partial solutions. -For finding these minimum cuts, two algorithms are presented. The +For finding the minimum cuts, two algorithms are presented. The push-relabel algorithm is considered to be the fastest and most versatile for general graphs, while the Boykov--Kolmogorov algorithm is specially taylored for the type of graphs we find in these kinds of imaging applications. -Further we have some results, and then that's it. +In the end we inspect restoration results to see how the method +performs. The construction of the tensor exposes three parameters, two +controlling the scale of the structures it should be sensitive to, and +one controlling the amount of anisotropy. We experiment with these +parameters to show that they do what we expect them to do. The method is +also compared with regular the total variation method to see that some +improvements are achieved. We also explore and look at the different +artifacts caused by approximations in the discretization, and how they +affect the restoration. %In everyday life cameras are used to capture a moment and save it for %eternity, but imaging technology technology can be used in many other @@ -534,7 +541,7 @@ functional decomposed as a sum over all the level values on the form + \beta \sum_{\lambda = 0}^{L-2} \sum_{(x, y)} F^{x,y}(u^\lambda_x, u^\lambda_y) =: \sum_{\lambda=0}^{L-2} F_\lambda(u^\lambda) - \label{eq:total_energy} + \label{eq:old_total_energy} \end{equation} where the sum over $(x, y)$ is over all pixel pairs $(x, y)$ in a neighbor relation, i.e.\ where the pixels are ``close'' to each other. diff --git a/main.tex b/main.tex index 357f623..e08df0b 100644 --- a/main.tex +++ b/main.tex @@ -204,6 +204,7 @@ Bjørn Rustad, \today. \input{discrete} \input{maxflow} \input{results} +\input{conclusion} \cleardoublepage diff --git a/results.tex b/results.tex index fbeffa1..0e2635e 100644 --- a/results.tex +++ b/results.tex @@ -198,7 +198,10 @@ anisotropy affected the regularization is not obvious however. ~ \begin{subfigure}[t]{0.30\textwidth} \centering - \includegraphics[width=\textwidth]{fig/factory/lines/r_p2_n8_b10000_g100_r4_s2_color.png} + \begin{overpic}[width=\textwidth]{fig/factory/lines/r_p2_n8_b10000_g100_r4_s2_color.png} + \put(80,0){\includegraphics[scale=0.05]{fig/wheel.png}} + \end{overpic} + %\includegraphics[width=\textwidth]{fig/factory/lines/r_p2_n8_b10000_g100_r4_s2_color.png} \caption{% Restored with relatively high noise scale of $\sigma = 2$. } @@ -207,7 +210,10 @@ anisotropy affected the regularization is not obvious however. ~ \begin{subfigure}[t]{0.30\textwidth} \centering - \includegraphics[width=\textwidth]{fig/factory/lines/r_p2_n8_b10000_g100_r15_s0.2_color.png} + \begin{overpic}[width=\textwidth]{fig/factory/lines/r_p2_n8_b10000_g100_r15_s0.2_color.png} + \put(80,0){\includegraphics[scale=0.05]{fig/wheel.png}} + \end{overpic} + %\includegraphics[width=\textwidth]{fig/factory/lines/r_p2_n8_b10000_g100_r15_s0.2_color.png} \caption{% Restored with low noise scale $\sigma = 0.2$, but large integration scale $\rho = 15$. @@ -262,7 +268,10 @@ favour of the larger, more coherent structure around it. ~ \begin{subfigure}[t]{0.30\textwidth} \centering - \includegraphics[width=\textwidth]{fig/factory/finger/r_p2_n32_b15000_g150_r10_s3_color.png} + \begin{overpic}[width=\textwidth]{fig/factory/finger/r_p2_n32_b15000_g150_r10_s3_color.png} + \put(80,80){\includegraphics[scale=0.05]{fig/wheel.png}} + \end{overpic} + %\includegraphics[width=\textwidth]{fig/factory/finger/r_p2_n32_b15000_g150_r10_s3_color.png} \caption{% $\rho = 10$ } @@ -270,14 +279,18 @@ favour of the larger, more coherent structure around it. ~ \begin{subfigure}[t]{0.30\textwidth} \centering - \includegraphics[width=\textwidth]{fig/factory/finger/r_p2_n32_b15000_g150_r20_s3_color.png} + \begin{overpic}[width=\textwidth]{fig/factory/finger/r_p2_n32_b15000_g150_r20_s3_color.png} + \put(80,80){\includegraphics[scale=0.05]{fig/wheel.png}} + \end{overpic} + %\includegraphics[width=\textwidth]{fig/factory/finger/r_p2_n32_b15000_g150_r20_s3_color.png} \caption{% $\rho = 20$ } \end{subfigure} \caption[Anisotropy tensor visualized for a fingerprint test image]{% A noisy fingerprint with the anisotropy tensor visualized for - different integration scales $\rho$. + different integration scales $\rho$. Parameters: + $\abs{\mathcal{N}} = 32$, $\sigma = 3$, $\beta = 15000$, $\omega = 150$. } \label{fig:finger_tensor} \end{figure} @@ -486,7 +499,11 @@ to small sized noise is shown in Table~\ref{tab:circumference}. The table shows how our discrete Cauchy--Crofton formula approximates the circumference of circles of different radii. Note that the circumference approximated is that of an actual continuous circle $u : \mathbb{R}^2 -\to \{0, 1\}$ and not a discrete representation. +\to \{0, 1\}$ and not a discrete representation. We see that the +perimeter of the smalles circle is grossly underestimated by the larger +neighborhoods. Thus the perimeter of one-pixel noise will be +underestimated, and in turn the contribution to the total variation by +one-pixel noise will be smaller for large neighborhoods. \section{Restoration} @@ -506,7 +523,7 @@ approximated is that of an actual continuous circle $u : \mathbb{R}^2 \centering \includegraphics[width=\textwidth]{fig/factory/finger/r_p2_n32_b15000_g150_r10_s3.png} \caption{% - Anisotropic TV ($\gamma = 150$), $\beta=15000$, + Anisotropic TV ($\omega = 150$), $\beta=15000$, $\norm{u - f}_{L^2} = 15272$. } \label{fig:finger_contrast_atv1} @@ -516,7 +533,7 @@ approximated is that of an actual continuous circle $u : \mathbb{R}^2 \centering \includegraphics[width=\textwidth]{fig/factory/finger/r_p2_n32_b20970_g150_r10_s3.png} \caption{% - Anisotropic TV ($\gamma = 150$), $\beta=20970$, + Anisotropic TV ($\omega = 150$), $\beta=20970$, $\norm{u - f}_{L^2} = 17022$. } \label{fig:finger_contrast_atv2} @@ -644,49 +661,3 @@ the image. % \label{fig:lena_method_noise} %\end{figure} -\chapter{Discussion and conclusion} - -In this thesis we have seen how the total variation restoration method -can be extended using an anisotropy tensor, and how this fits into the -discretization and graph cut framework used in my project work -\cite{project}. The anisotropy tensor was introduced in hopes of -reducing the amount of regularization across edges in the image. It was -constructed based on the structure tensor with three parameters exposed -for controlling the method. - -The continuous functional we initally wanted to minimize was then -transformed using an anisotropic coarea formula and an anisotropic -Cauchy--Crofton formula, both described in detail. The functional was -discretized, and the discretization was shown---under some -restrictions---to be consistent with the continuous functional. - -As in my project work \cite{project} the discrete functional was -minimized using maximum flow algorithms to obtain successive minimum -graph cuts. The description of the push-relabel algorithm was included from -my project work for completeness, and a description of the -Boykov--Kolmogorov algorithm was also given, as it is taylored for these -kinds of graph cut applications. - -Because we have spread our attention across all parts of the restoration -method, it is also possible to go deeper into the theory behind all of -them. The well-foundedness of the continuous formulation and the -anisotropic coarea and Cauchy--Crofton formula could have been studied -further on a measure-theoretic foundation. Further, in the -discretization there are a myriad of choices that could have been -discussed further, most notably the construction of the neighborhood -stencil, which has a lot to say for the performance, and approximation -error of the discrete solution. Further, since we start in the discrete -setting with a digital image, and obtain a guaranteed minimizer of the -discrete functional, we did not discuss stability and convergence in -relation to a theoretical solution to the continuous problem. - -In the end we looked at how the different parameters affect the -performance of the algorithm. We will not give any unified conclusion as -to whether this method is ``better'' or ``worse'' than the regular total -variation method it is based on, or other methods. In different -applications the input images have different properties, and one might -also have different hopes for, and restrictions on the output image. -Thus I hope a thorough description in addition to an inspection of -restoration results has given insight into the good and not-so-good sides -of this particular method. -