From: Bjørn Rustad Date: Thu, 22 May 2014 11:14:15 +0000 (+0200) Subject: Fixes from Markus X-Git-Url: http://git.rustad.me/?a=commitdiff_plain;h=866fbedbbdc8c0ba65fdab1f8d5d7f32bfdd3e80;p=prosjektoppgave Fixes from Markus --- diff --git a/bib.bib b/bib.bib index f48d0a4..bd84799 100644 --- a/bib.bib +++ b/bib.bib @@ -344,7 +344,7 @@ incollection{dinitz2006dinitz, @article{cheriyan1989analysis, title={Analysis of preflow push algorithms for maximum network flow}, - author={Cheriyan, Joseph and Maheshwari, SN}, + author={Cheriyan, Joseph and Maheshwari, S. N.}, journal={SIAM Journal on Computing}, volume={18}, number={6}, diff --git a/fig/img_decomp.tex b/fig/img_decomp.tex index 634fab2..a172f98 100644 --- a/fig/img_decomp.tex +++ b/fig/img_decomp.tex @@ -130,7 +130,7 @@ to[out=180,in=-50] (4,7.25); \end{tikzpicture} - \caption{The thresholded images, which are in + \caption{The thresholded images, which are elements of $\{0,1\}^{\abs{\Omega}}$.} \end{subfigure} \caption{A sample gray-scale image, and its decomposition into diff --git a/methods.tex b/methods.tex index dc2a6f3..b7b726b 100644 --- a/methods.tex +++ b/methods.tex @@ -12,7 +12,7 @@ method where the image is convolved with the Gaussian function G_\sigma(x, y) = \frac{1}{2 \pi \sigma^2}\exp\left(-\frac{x^2 + y^2}{2\sigma^2}\right). \end{equation} -In the discrete case of an image consisting separate pixels, Gaussian +In the discrete case of an image consisting of separate pixels, Gaussian blur amounts to setting each pixel value as the weighted average of its neighbors in the original image. This will, in addition to smoothing out possible noise, add blur and remove details from the image. @@ -71,7 +71,7 @@ tensor $A(u)$ in the equation \partial_t u = \mathrm{div} \big(A(u) \cdot \nabla u\big). \end{equation} This way, it is possible to have a small diffusivity \emph{across} -edges, and at the same time a big diffusivity \emph{along} the edges. +edges, and at the same time a large diffusivity \emph{along} the edges. See for example \cite{weickert1998anisotropic} for more information on these diffusion tensor methods. diff --git a/total.tex b/total.tex index dcf38c2..409d82a 100644 --- a/total.tex +++ b/total.tex @@ -35,8 +35,10 @@ take different physical phenomena into account. We will in this brief introduction only consider the simplest model, namely \emph{additive noise}, which given an image $u$ results in a captured image \begin{equation} - v = u + \epsilon. + v = u + \epsilon, \end{equation} +where $\epsilon$ represents the noise that can be modeled by different +probability distributions. For a captured noisy image $v$ and we want to recover $u$, or something as close to $u$ as possible, so we introduce the likelihood function @@ -55,12 +57,12 @@ increasing function, the maximization problem is equivalent to \min_u \big( - \ln p(v \mid u) - \ln p(u) \big). \label{eq:minimize_neg_log} \end{equation} -For discrete images consisting of pixels, we denote use $x$ and $y$ to -denote pixel positions such that $u_x$ and $u_y$ are the values of those -pixels, not to be confused with partial derivatives $\partial_x u$ and -$\partial_y u$. If the noise term $p(v \mid u)$ is modeled to be -independently and identically distributed Gaussian noise on each pixel, -we have +For discrete images consisting of pixels, we use $x$ and $y$ to denote +pixel positions such that $u_x$ and $u_y$ are the values of those +pixels, \emph{not to be confused with} partial derivatives $\partial_x +u$ and $\partial_y u$ which will not be used. If the noise term $p(v +\mid u)$ is modeled to be independently and identically distributed +Gaussian noise on each pixel, we have \begin{equation} p(v \mid u) \propto \prod_x \exp \left(-\frac{(u_x - v_x)^2}{2\sigma^2} \right). @@ -87,11 +89,12 @@ Inserting the prior \eqref{eq:laplace_prior} and the noise model \begin{equation} \min_u \left( \sum_x \abs{u_x - v_x}^2 - - \beta \sum_x \sum_{y \in \mathcal{N}(x)} \abs{u_x - u_y} - \right). + + \beta \sum_x \sum_{y \in \mathcal{N}(x)} \abs{u_x - u_y} + \right), \label{eq:minimizing_probability_l2} \end{equation} -If the noise is modeled by a Laplace distribution +where $\beta$ is a combination of the coefficients in the two terms. If +the noise is modeled by a Laplace distribution \begin{equation} p(u \mid v) \propto \prod_x \exp \left( - \frac{\abs{u_x - v_x}}{b} \right) @@ -100,7 +103,7 @@ we obtain the $L^1$-norm instead of the $L^2$-norm, \begin{equation} \min_u \left( \sum_x \abs{u_x - v_x} - - \beta \sum_x \sum_{y \in \mathcal{N}(x)} \abs{u_x - u_y} + + \beta \sum_x \sum_{y \in \mathcal{N}(x)} \abs{u_x - u_y} \right). \label{eq:minimizing_probability_l1} \end{equation} @@ -123,7 +126,7 @@ minimizing the energy function \begin{equation} E_v(u) = \int_\Omega \abs{u - v}^p - + \beta \TV(u) + + \beta\; \TV(u) \label{eq:energy_function} \end{equation} where $v$ is the original image, and $\Omega$ is the image domain, in @@ -133,15 +136,15 @@ models in \eqref{eq:minimizing_probability_l1} and \eqref{eq:minimizing_probability_l2}. In the following, assume that our image domain $\Omega$ is an open and -bounded subset of $\mathbb{R}^2$, with a Lipschitz boundary. Since most -images are given on rectangles in $\mathbb{R}^2$, this is not -problematic. +bounded subset of $\mathbb{R}^2$ with a Lipschitz boundary. Since most +images are given on rectangles in $\mathbb{R}^2$, this is not a +problem. The term $\TV(u)$ in \eqref{eq:energy_function} is the total variation of the image, introduced earlier as $\int_\Omega \abs{\nabla u}$. Since -it is problematic to calculate the traditional gradient at -discontinuities, the following definition is introduced, using Green's -identity to transfer the derivative to a smooth test function. +it is problematic to calculate the traditional gradient, the following +definition is introduced, using Green's identity to transfer the +derivative to a smooth test function. \begin{definition}[Total variation] Given a function $u \in L^1(\Omega)$, the total variation of $u$, often written $\int_\Omega \abs{Du}$, where the $D$ is the gradient @@ -160,8 +163,8 @@ identity to transfer the derivative to a smooth test function. $\Omega$. \end{definition} Note that since $\Omega$ is open and bounded, the test functions -$\varphi$ never has support on the boundary of $\Omega$. Next we -introduce the space of functions with bounded variation. +$\varphi$ vanish on the boundary of $\Omega$. Next we introduce the +space of functions with bounded variation. \begin{definition}[Functions of bounded variation] The space of functions of bounded variation $\BV(\Omega)$ is the space of functions $u \in L^1(\Omega)$ for which the total variation @@ -171,17 +174,17 @@ introduce the space of functions with bounded variation. \right\}. \end{equation} \end{definition} -When discretizing the energy function \eqref{eq:energy_function} later -on, we want to decompose it as a sum over the different levels, or pixel -values, of the image. This is why we present the coarea formula here, -and use it to write the total variation $\TV(u)$ as an integral over the -range level values $\lambda$. The coarea formula and further references -can be found in \cite{caselles2011total}. - -First we need to introduce the formal definition of a set perimeter. -Here, $\idfun_E$ signifies the characteristic function of the set $E$, a -function which is equal to one in every point in $E$, and zero -elsewhere. +%When discretizing the energy function \eqref{eq:energy_function} later +%on, we want to decompose it as a sum over the different levels, or pixel +%values, of the image. This is why we will present the coarea formula, +%and use it to write the total variation $\TV(u)$ as an integral over the +%range of the level values $\lambda$. The coarea formula and further +%references can be found in \cite{caselles2011total}. + +Before stating the coarea formula, we need the formal definition of a +set perimeter. Here, $\idfun_E$ signifies the characteristic function of +the set $E$, the function which is equal to one in every point in $E$, +and zero elsewhere. \begin{definition}[Set perimeter] The perimeter of a set $E$ is defined as \begin{equation} @@ -197,19 +200,23 @@ $P(E,\Omega)$ will not include the overlapping part of the boundary. \fixme{Bad! The level sets are not sets! But the perimeter function want sets!} -For an image $u$ we introduce the level sets of $u$ as +For an image $u$ we denote by $u \leq \lambda$ the set $\{ x \in \Omega +: u_x \leq \lambda\}$ and the thresholded images \begin{equation} - u^\lambda = \idfun_{u \leq \lambda}, + u^\lambda = \idfun_{u \leq \lambda}. \label{eq:level_set} \end{equation} -and with that we are ready to introduce the coarea formula. +With all these definitions in place, we are ready to introduce the +coarea formula, which will let us write the total variation as an +integral over the range of the image levels $\lambda$. \begin{theorem}[The coarea formula] Let $u \in BV(\Omega)$. Then for almost every $\lambda$ the set - $u^\lambda$ is of finite perimeter, and one has the coarea formula + $u \leq \lambda$ is of finite perimeter, and one has the coarea + formula \begin{equation} \TV(u) = \int_\Omega \abs{D u} - = \int_\mathbb{R} P(u^\lambda,\Omega) \, d\lambda + = \int_\mathbb{R} P(u \leq \lambda,\Omega) \, d\lambda \label{eq:coarea_formula} \end{equation} \end{theorem} @@ -218,22 +225,26 @@ rewrite \eqref{eq:energy_function} as \begin{equation} E_v(u) = \int_\Omega \abs{u - v}^p - + \beta \int_\mathbb{R} P(u^\lambda,\Omega) \, d\lambda, + + \beta \int_\mathbb{R} P(u \leq \lambda,\Omega) \, d\lambda. \label{eq:energy_perimeter} \end{equation} -which is a good starting point for the discretization step. More -analysis of the continuous problem is given in \cite{caselles2011total}, -together with different numerical methods for solving it. Results on how -the set of jumps in the resulting image $u$ is contained in the set of -jumps in the original image $v$ are also presented. +This will be the starting point for our discretization, and it will +allow us to decompose the energy into a sum over the discrete set of +level values. + +More analysis of the continuous problem is given in +\cite{caselles2011total}, together with different numerical methods for +solving it. Results on how the set of jumps in the resulting image $u$ +is contained in the set of jumps in the original image $v$ are also +presented there. \subsection{Discrete formulation} -Since digital images are given on a discrete grid, with values taken -from a discrete and finite set of levels, we want to discretize +Since digital images are given on a discrete grid with values taken from +a discrete and finite set of levels, we want to discretize \eqref{eq:energy_perimeter}, and solve the resulting discrete problem. -We denote $S$ for the grid of pixels, and assume that the value of each -pixel is taken from the set $\mathcal{L} = \{0, \hdots, L-1\}$. This is -a reasonable assumption for grayscale images. +We denote $\mathcal{G}$ for the grid of pixels, and assume that the +value of each pixel is taken from the set $\mathcal{L} = \{0, \hdots, +L-1\}$. This is a reasonable assumption for grayscale images. \subsubsection{Regularization term} Chambolle discretizes the total variation as @@ -248,19 +259,18 @@ introduces his dual approach to the continuous minimization problem. This straight-forward way of discretizing the gradient offers, according to Chambolle, a good compromize between isotropy and stability. However, it does not decompose into a sum over the levels of the image, which we -need for our graph composition. +need for our graph construction. \begin{figure} \input{fig/img_decomp} \end{figure} Following the notation used in \cite{darbon2006image}, we let $u_x$ -denote the value of the image $u$ at position $x \in S$. With a finite -set of pixel values $\mathcal{L}$, we also have a finite number of level -sets $u^\lambda$. Figure \ref{fig:img_decomp} visualizes -a possible image and its decomposition into thresholded images, one for -each level value. -\fixme{Position $x$ already used in stats section. Problem?} +denote the value of the image $u$ at position $x \in \mathcal{G}$. With +a finite set of pixel values $\mathcal{L}$, we also have a finite number +of thresholded images $u^\lambda$. Figure \ref{fig:img_decomp} +visualizes a possible image and its decomposition into thresholded +images, one for each level value. \begin{figure} \centering @@ -350,16 +360,16 @@ setting, such that the total variation can be written as a sum over the level values \begin{equation} \TV(u) - = \sum_{\lambda = 0}^{L-2} P(u^\lambda, S) - = \sum_{\lambda = 0}^{L-2} \sum_{(x,y)} w_{xy} + = \sum_{\lambda = 0}^{L-2} P(u \leq \lambda, \Omega) + \approx \sum_{\lambda = 0}^{L-2} \sum_{(x,y)} w_{xy} \abs{u_x^\lambda - u_y^\lambda}. \label{eq:tv_discrete_int} \end{equation} The sum over $(x,y)$ signifies a sum over all pixels cliques of size two, i.e.\ all pairs $(x, y)$ such that $x$ and $y$ are in a -neighborhood relation. The $w_{xy}$ is a weight parameter. The sum over -$\lambda$ ends at $L-2$ since $u^{L-1}$ is equal to 1 in every pixel of -the image. +neighborhood relation. The $w_{xy}$ is a weight parameter, and the sum +over $\lambda$ ends at $L-2$ since $u^{L-1}$ is equal to 1 in every +pixel of the image. Intuitively, the perimeter of the level set $u^\lambda$ is proportional to the number of pixels at the boundary of the set. This is again @@ -375,23 +385,12 @@ weight is chosen as the discrete perimeter in \eqref{eq:tv_discrete_int} converges to the continuous perimeter in \eqref{eq:coarea_formula}. Here, $h$ is the grid size and $d_{xy}$ is the euclidean distance of the edge from $x$ to $y$. -The $\Delta \phi_{xy}$ is the difference between the angle of this edge -and the next edge, if the edges are sorted by increasing angles. These -parameters are also shown in Figure \ref{fig:common_neighborhoods}. -Boykov and Kolmogorov prove we have convergence when all of $h$, $\Delta -\phi_{xy}$, and $d_{xy}$ go to zero. - -\fixme{IS THIS EVEN NECESSARY? For two binary variables $a$ and $b$ we - can easily verify that $\abs{a - b} = a + b - 2 a b$. Using this we - rewrite \eqref{eq:tv_discrete_int} to - \begin{equation} - \TV(u) = \sum_{\lambda = 0}^{L-2} \sum_{(x,y)} w_{xy} - \left( - \left(1 - 2 u_y^\lambda \right) u_x^\lambda + u_y^\lambda - \right). - \label{eq:tv_discrete} - \end{equation} -} +The number $\Delta \phi_{xy}$ is the difference between the angle of +this edge and the next edge, if the edges are sorted by increasing +angles. These parameters are also shown in Figure +\ref{fig:common_neighborhoods}. Boykov and Kolmogorov prove we have +convergence when all of $h$, $\Delta \phi_{xy}$, and $d_{xy}$ go to +zero. \subsubsection{Fidelity term} Now that the total variation is discretized, we need to take care of the @@ -403,7 +402,6 @@ value of the energy if we were to color pixel $x$ with label $k$ \end{equation} This allows us to discretize $\int_\Omega \abs{u - v}^p$ of \eqref{eq:energy_perimeter} in the following way -\fixme{Give this a name?} \begin{equation} \sum_x \abs{u_x - v_x}^p = \sum_x N_x(u_x) \label{eq:norm_discrete_int}. @@ -433,8 +431,6 @@ Since $\mathbbm{1}_{\lambda < u_x} = (1 - u^\lambda_x)$ we rewrite (1 - u^\lambda_x) + N_x(0) \end{equation} -\fixme{$E_\lambda^x$} - \subsubsection{Total energy} \label{sec:total_energy} @@ -443,7 +439,7 @@ over all the levels $\lambda$. Ignoring the constant term $N_x(0)$ we are left with \begin{equation} E_v(u) = - \sum_{\lambda=0}^{L-2} \sum_x E^x(u^\lambda_x) + \sum_{\lambda=0}^{L-2} \sum_x E^x_\lambda(u^\lambda_x) + \beta \sum_{\lambda = 0}^{L-2} \sum_{(x, y)} E^{x,y}(u^\lambda_x, u^\lambda_y) = \sum_{\lambda=0}^{L-2} F_\lambda(u^\lambda) @@ -451,7 +447,7 @@ are left with \end{equation} where \begin{align} - E^x(u^\lambda_x) &= + E^x_\lambda(u^\lambda_x) &= \big( N_x(\lambda + 1) - N_x(\lambda) @@ -473,15 +469,15 @@ set $u^\lambda$ that minimizes $F_\lambda(u^\lambda)$, the sum in these obtained level sets $u^\lambda$ can be combined to make an output image. The level sets were defined as $u^\lambda = \mathbbm{1}_{u \leq \lambda}$, so we need them to be monotonically increasing in increasing -level values, i.e.\ +level values, i.e.\, \begin{equation} u^\lambda_x \leq u^\mu_x \quad \forall \lambda \leq \mu, \quad \forall x \in S. \end{equation} In the following sections we will present a graph cut algorithm that -finds thresholded images minimizing each level, \emph{while} -guaranteeing that they meet this requirement. +finds thresholded images minimizing each level, \emph{while guaranteeing +that they meet this requirement.} \fixme{Consistently call them the fidelity and regularization terms.}