From: Bjørn Rustad Date: Wed, 26 Nov 2014 18:18:10 +0000 (+0100) Subject: MOEAR X-Git-Url: http://git.rustad.me/?a=commitdiff_plain;h=8ff1869027416edb54c11018ef2c1607180b885c;p=master MOEAR --- diff --git a/bib.bib b/bib.bib index 009fec9..f3efebf 100644 --- a/bib.bib +++ b/bib.bib @@ -99,6 +99,13 @@ publisher={Elsevier} } +@article{project, + title={Total variation based image restoration using graph cuts}, + author={Rustad, Bjørn}, + year={2014}, + publisher={NTNU} +} + @article{darbon2006image, AUTHOR = {Darbon, J{\'e}r{\^o}me and Sigelle, Marc}, TITLE = {Image restoration with discrete constrained total variation. diff --git a/fig/edges.pgm b/fig/edges.pgm new file mode 100644 index 0000000..7ab4a35 Binary files /dev/null and b/fig/edges.pgm differ diff --git a/fig/edges.png b/fig/edges.png new file mode 100644 index 0000000..8169a49 Binary files /dev/null and b/fig/edges.png differ diff --git a/introduction.tex b/introduction.tex index 973be50..696d40f 100644 --- a/introduction.tex +++ b/introduction.tex @@ -119,6 +119,7 @@ becomes \end{cases} \label{eq:aniso_diff} \end{equation} +\fixme{triple column alignment?} where $\nu$ is the outer normal of $\Omega$. The tensor $A(u)$ would normally be constructed such as to diminish the effect of $\nabla u$ across what we believe to be edges in the image. This way, there will @@ -219,6 +220,7 @@ variation using the distributional derivative. $C^\infty_c\left(\Omega, \mathbb{R}^2\right)$, the space of smooth functions from $\Omega$ to $\mathbb{R}^2$ with compact support in $\Omega$. + \label{def:tv} \end{definition} Note that since $\Omega$ is open and bounded, the test functions diff --git a/theory.tex b/theory.tex index ef95f59..5ce6755 100644 --- a/theory.tex +++ b/theory.tex @@ -22,8 +22,8 @@ the anisotropic total variation \label{eq:aniso_tv_sqrt} \end{equation} for all $u \in C^1(\Omega)$. If $A(x)$ is the identity -matrix we get the regular total variation found in -\eqref{first_min_presentation}. One problem with the regular total +matrix we get the regular total variation found in Definition +\ref{def:tv}. One problem with the regular total variation method is that when reducing the total variation, it will also try to reduce the variation over known edges in the image, which can lead to contrast loss, especially in thin details. By controlling $A(x)$ @@ -69,6 +69,7 @@ With this extended definition, we have arrived at a minimization problem where we seek to find a minimizer of the functional \begin{equation} F(u) = \int_\Omega (u - f)^2 \, dx + \beta \, \TVA(u) + \label{eq:first_anisotropic_functional} \end{equation} The question is then how to construct this anisotropy tensor $A(x)$ to get the improvements we hope for, and how this affects our numerical @@ -107,7 +108,8 @@ none, so we somehow want to be sure about the edges we find. Edges can be found in many different ways, but as suggested by Weickert in his book on Anisotropic Diffusion \cite{weickert1998anisotropic}, and briefly mentioned in Section \ref{sec:anisotropic_diffusion}, a good -starting point is the structure tensor +starting point is the \emph{edge detector} $\nabla \tilde{f}_\sigma$ and +the structure tensor \begin{equation} S_\rho(x) := K_\rho * \left( \nabla \tilde{f}_\sigma(x) \otimes \nabla \tilde{f}_\sigma(x) \right), @@ -120,11 +122,23 @@ parameter $\sigma$, called the \emph{noise scale} by Weickert, will control how sensitive the edge detector $\nabla \tilde{f}_\sigma$ is to noise and small details. +\begin{figure} + \centering + \includegraphics[width=0.4\textwidth]{fig/edges.png} + \caption{ + The largest eigenvalue of the structure tensor is $\abs{\nabla + \tilde{f}_\sigma(x)}^2$ and is as we see a good edge detector. + } + \label{fig:edges} +\end{figure} + First consider the tensor $S_0(x) = \nabla \tilde{f}_\sigma(x) \otimes \nabla \tilde{f}_\sigma(x)$. It obviously contains the same information as the edge detector itself. Its eigenvalues will be 0 and $\labs{\nabla \tilde{f}_\sigma(x)}^2$ with corresponding eigenvectors $v_1$ and $v_2$ perpendicular and parallel to $\nabla \tilde{f}_\sigma(x)$ repectively. +Figure \ref{fig:edges} shows that the largest eigenvalue of the +structure tensor is a good edge detector. We are also interested in identifying and being sensitive to features in a neighborhood around the point, such as corners or curved edges and @@ -137,7 +151,17 @@ structures we want our anisotropy tensor to be sensitive to. Being a smoothed combination of symmetric positive semi-definite matrices, $S_\rho(x)$ can easily be verified to be symmetric positive semi-definite itself. We order its two real eigenvalues $\lambda_1 \geq -\lambda_2$ with corresponding eigenvectors $v_1$ and $v_2$. The vector +\lambda_2$ with corresponding eigenvectors $v_1$ and $v_2$. From the +characteristic polynomial of $S_\rho(x) = (\begin{smallmatrix} s_{11} & +s_{12} \\ s_{12} & s_{22} \end{smallmatrix})$ we obtain a closed form +expression for the eigenvalues +\begin{equation} + \lambda = \frac{1}{2} \left( s_{11} + s_{22} \pm \sqrt{(s_{11} - + s_{22})^2 + 4 s_{12}^2} \right) + \label{eq:s_eigenvalues} +\end{equation} + +The vector $v_1$ will then indicate the direction of most variation in the neighborhood. From this we see that an edge would give $\lambda_1 \gg \lambda_2 \approx 0$, while smooth areas would give $\lambda_1 \approx @@ -145,9 +169,9 @@ neighborhood. From this we see that an edge would give $\lambda_1 \gg $v_1$ but also perpendicular to $v_1$, so we will have $\lambda_1 \approx \lambda_2 \gg 0$. -These eigenvalues indicate that the product $\sqrt{\nabla u^T S_\rho(x) -\nabla u}$ would amplify the effect of $\nabla u$ across edges, the -opposite of what we want. +These eigenvalues indicate that the product $\nabla u^T S_\rho(x) \nabla +u$ would amplify the effect of $\nabla u$ across edges, the opposite of +what we want. We begin our anisotropy tensor construction by eigendecomposing the structure tensor as @@ -175,11 +199,14 @@ U(x) \Sigma(x) U(x)^T$ where \end{equation} To reverse the effect of $S_\rho(x)$ such that $A(x)$ actually weighs down the norm across edges, we need $\sigma_1 \leq \sigma_2$, so we construct -\begin{align} - \sigma_1 &= \left(1 + \frac{(\lambda_1 - - \lambda_2)^2}{\gamma^2}\right)^{-1}, \\ - \sigma_2 &= 1. -\end{align} +\begin{equation} + \begin{aligned} + \sigma_1 &= \left(1 + + \frac{(\lambda_1 - \lambda_2)^2}{\gamma^2}\right)^{-1}, \\ + \sigma_2 &= 1. + \end{aligned} + \label{eq:sigma_construction} +\end{equation} This way, $\sigma_1 \in (0, 1]$. The rotation is kept, while the size of the eigenvalues are changed. This should be visualized, with a figure showing the length and @@ -325,7 +352,8 @@ semi-continous, we present a much needed result. which proves that $F$ is sequentially weakly lower semi-continuous. \end{proof} -From our functional in \fixme{ref}, we first consider the fidelity term +From our functional in \eqref{eq:first_anisotropic_functional}, we first +consider the fidelity term \begin{equation} \int_\Omega (u - v)^2 \, dx %= \sup_{\substack{\xi \in L^2(\Omega) \\ \norm{\xi}_{L^2} \leq @@ -390,6 +418,7 @@ given in \cite{olsson2009extending}. can be written as an integral over all the levels \begin{equation} \TVA(u) = \int_{-\infty}^\infty \TVA(u^s) \, ds. + \label{eq:anisotropic_coarea} \end{equation} \label{thm:anisotropic_coarea} \end{theorem} @@ -522,7 +551,11 @@ given in \cite{olsson2009extending}. This coarea formula is our first step in transforming the anisotropic total variation into an easily discretizisable expression. -This is where we introduce the perimeter, or more specifically +The anisotropic total variation of the thresholded images occuring in +the anisotropic coarea formula are very much related to the size of the +boundary of the level set, as the only variation in a characteristic +function, occurs at the boundary of the set. This is why we introduce +the following definition of the anisotropic set perimeter. \begin{definition}[The anisotropic set perimeter] Given an anisotropy tensor $A$ the anisotropic perimeter of a set $U$ is defined as @@ -977,25 +1010,30 @@ The structure tensor is constructed as described in Section \end{equation} Because of the convolutions with the Gaussian function, this is a smooth continuous map from $\bar{\Omega}$ to $\mathbb{R}^{2 \times 2}$. As we -can see in \fixme{ref} the eigenvalues are the roots of a monic -polynomial and thus depend continuously on the coefficients of the -polynomial, which in turn are continuous functions of the elements in +can see in \eqref{eq:s_eigenvalues}, the eigenvalues +depend continuously on the coefficients of the +elements in the structure tensor $S_\rho(x)$. The extreme value theorem \fixme{ref} states that a continuous real-valued function on a nonempty compact -space is bounded above. Thus the eigenvalues $s_1$ and $s_2$ of $S_\rho(x)$ -are bounded from above and by construction the smallest eigenvalue of +space is bounded above. Thus the eigenvalues $\lambda_1$ and $\lambda_2$ of $S_\rho(x)$ +are bounded from above and by the construction in +\eqref{eq:sigma_construction}, the smallest eigenvalue of our anisotropy tensor $A(x)$ is bounded away from zero as \begin{equation} \lambda_2 = \left(1 + \frac{(s_1 - s_2)^2}{\gamma^2}\right)^{-1} \geq \left(1 + - \frac{s_1^2}{\gamma^2}\right)^{-1} \geq Q > 0. + \frac{s_1^2}{\gamma^2}\right)^{-1} \geq k > 0. \end{equation} \fixme{bah, now gamma is a curve...} -Hence, our metric tensor $M(x) = P A(x) P^T$ in -\eqref{eq:per_to_length2} fulfills all the assumptions of Theorem +Hence, our metric tensor $M(x) = P A(x) P^T$ and curve length +calculation in +\eqref{eq:per_to_length2} fulfill all the assumptions of Theorem \ref{thm:riemannian_cauchy_crofton}. \fixme{did we argue for -continuous?} Inserting the Cauchy--Crofton formula, we obtain the final -expression for the continuous anisotropic total variation +continuous?} Thus we can apply the Cauchy--Crofton formula of +\eqref{eq:riemannian_cauchy_crofton} to calculate the curve length in +\eqref{eq:per_to_length2} and through the anisotropic coarea formula in +Theorem \ref{thm:anisotropic_coarea} obtain our final expression for the +continuous anisotropic total variation \begin{equation} \TVA(u) = \int_{-\infty}^{\infty} \int_\mathcal{L} \sum_{x \in \ell_{\nu, \rho} \cap \gamma_s } \frac{\det M(x)} @@ -1009,7 +1047,8 @@ $\det A = \det PAP^T = \det M$, and from our eigendecomposition in amounts to switching the two eigenvalues $\sigma_1$ and $\sigma_2$ in $\Sigma$. -The whole transformation from the initial formulation in \fixme{ref} +The whole transformation from the initial functional in +\eqref{eq:first_anisotropic_functional} through the coarea formula and the Cauchy--Crofton formula was motivated by the discretization wich will be described next. We will see that we can minimize the energy functional for each level separately, and that @@ -1036,7 +1075,8 @@ energyfunction as a sum over all the levels $\mathcal{L}$. \subsection{Fidelity term} -For some pixel position $x \in \mathcal{G}$ and some level value $k \in +The fidelity term can be discretize as in my project work +\cite{project}. For some pixel position $x \in \mathcal{G}$ and some level value $k \in \mathcal{L}$, we define the following function \begin{equation} N_x(k) = \abs{k - v_x}^p