introduction only consider the simplest model, namely \emph{additive
noise}, which given an image $u$ results in a captured image
\begin{equation}
- v = u + \epsilon.
+ v = u + \epsilon,
\end{equation}
+where $\epsilon$ represents the noise that can be modeled by different
+probability distributions.
For a captured noisy image $v$ and we want to recover $u$, or something
as close to $u$ as possible, so we introduce the likelihood function
\min_u \big( - \ln p(v \mid u) - \ln p(u) \big).
\label{eq:minimize_neg_log}
\end{equation}
-For discrete images consisting of pixels, we denote use $x$ and $y$ to
-denote pixel positions such that $u_x$ and $u_y$ are the values of those
-pixels, not to be confused with partial derivatives $\partial_x u$ and
-$\partial_y u$. If the noise term $p(v \mid u)$ is modeled to be
-independently and identically distributed Gaussian noise on each pixel,
-we have
+For discrete images consisting of pixels, we use $x$ and $y$ to denote
+pixel positions such that $u_x$ and $u_y$ are the values of those
+pixels, \emph{not to be confused with} partial derivatives $\partial_x
+u$ and $\partial_y u$ which will not be used. If the noise term $p(v
+\mid u)$ is modeled to be independently and identically distributed
+Gaussian noise on each pixel, we have
\begin{equation}
p(v \mid u) \propto
\prod_x \exp \left(-\frac{(u_x - v_x)^2}{2\sigma^2} \right).
\begin{equation}
\min_u \left(
\sum_x \abs{u_x - v_x}^2
- - \beta \sum_x \sum_{y \in \mathcal{N}(x)} \abs{u_x - u_y}
- \right).
+ + \beta \sum_x \sum_{y \in \mathcal{N}(x)} \abs{u_x - u_y}
+ \right),
\label{eq:minimizing_probability_l2}
\end{equation}
-If the noise is modeled by a Laplace distribution
+where $\beta$ is a combination of the coefficients in the two terms. If
+the noise is modeled by a Laplace distribution
\begin{equation}
p(u \mid v) \propto
\prod_x \exp \left( - \frac{\abs{u_x - v_x}}{b} \right)
\begin{equation}
\min_u \left(
\sum_x \abs{u_x - v_x}
- - \beta \sum_x \sum_{y \in \mathcal{N}(x)} \abs{u_x - u_y}
+ + \beta \sum_x \sum_{y \in \mathcal{N}(x)} \abs{u_x - u_y}
\right).
\label{eq:minimizing_probability_l1}
\end{equation}
\begin{equation}
E_v(u) =
\int_\Omega \abs{u - v}^p
- + \beta \TV(u)
+ + \beta\; \TV(u)
\label{eq:energy_function}
\end{equation}
where $v$ is the original image, and $\Omega$ is the image domain, in
\eqref{eq:minimizing_probability_l2}.
In the following, assume that our image domain $\Omega$ is an open and
-bounded subset of $\mathbb{R}^2$, with a Lipschitz boundary. Since most
-images are given on rectangles in $\mathbb{R}^2$, this is not
-problematic.
+bounded subset of $\mathbb{R}^2$ with a Lipschitz boundary. Since most
+images are given on rectangles in $\mathbb{R}^2$, this is not a
+problem.
The term $\TV(u)$ in \eqref{eq:energy_function} is the total variation
of the image, introduced earlier as $\int_\Omega \abs{\nabla u}$. Since
-it is problematic to calculate the traditional gradient at
-discontinuities, the following definition is introduced, using Green's
-identity to transfer the derivative to a smooth test function.
+it is problematic to calculate the traditional gradient, the following
+definition is introduced, using Green's identity to transfer the
+derivative to a smooth test function.
\begin{definition}[Total variation]
Given a function $u \in L^1(\Omega)$, the total variation of $u$,
often written $\int_\Omega \abs{Du}$, where the $D$ is the gradient
$\Omega$.
\end{definition}
Note that since $\Omega$ is open and bounded, the test functions
-$\varphi$ never has support on the boundary of $\Omega$. Next we
-introduce the space of functions with bounded variation.
+$\varphi$ vanish on the boundary of $\Omega$. Next we introduce the
+space of functions with bounded variation.
\begin{definition}[Functions of bounded variation]
The space of functions of bounded variation $\BV(\Omega)$ is the
space of functions $u \in L^1(\Omega)$ for which the total variation
\right\}.
\end{equation}
\end{definition}
-When discretizing the energy function \eqref{eq:energy_function} later
-on, we want to decompose it as a sum over the different levels, or pixel
-values, of the image. This is why we present the coarea formula here,
-and use it to write the total variation $\TV(u)$ as an integral over the
-range level values $\lambda$. The coarea formula and further references
-can be found in \cite{caselles2011total}.
-
-First we need to introduce the formal definition of a set perimeter.
-Here, $\idfun_E$ signifies the characteristic function of the set $E$, a
-function which is equal to one in every point in $E$, and zero
-elsewhere.
+%When discretizing the energy function \eqref{eq:energy_function} later
+%on, we want to decompose it as a sum over the different levels, or pixel
+%values, of the image. This is why we will present the coarea formula,
+%and use it to write the total variation $\TV(u)$ as an integral over the
+%range of the level values $\lambda$. The coarea formula and further
+%references can be found in \cite{caselles2011total}.
+
+Before stating the coarea formula, we need the formal definition of a
+set perimeter. Here, $\idfun_E$ signifies the characteristic function of
+the set $E$, the function which is equal to one in every point in $E$,
+and zero elsewhere.
\begin{definition}[Set perimeter]
The perimeter of a set $E$ is defined as
\begin{equation}
\fixme{Bad! The level sets are not sets! But the perimeter
function want sets!}
-For an image $u$ we introduce the level sets of $u$ as
+For an image $u$ we denote by $u \leq \lambda$ the set $\{ x \in \Omega
+: u_x \leq \lambda\}$ and the thresholded images
\begin{equation}
- u^\lambda = \idfun_{u \leq \lambda},
+ u^\lambda = \idfun_{u \leq \lambda}.
\label{eq:level_set}
\end{equation}
-and with that we are ready to introduce the coarea formula.
+With all these definitions in place, we are ready to introduce the
+coarea formula, which will let us write the total variation as an
+integral over the range of the image levels $\lambda$.
\begin{theorem}[The coarea formula]
Let $u \in BV(\Omega)$. Then for almost every $\lambda$ the set
- $u^\lambda$ is of finite perimeter, and one has the coarea formula
+ $u \leq \lambda$ is of finite perimeter, and one has the coarea
+ formula
\begin{equation}
\TV(u)
= \int_\Omega \abs{D u}
- = \int_\mathbb{R} P(u^\lambda,\Omega) \, d\lambda
+ = \int_\mathbb{R} P(u \leq \lambda,\Omega) \, d\lambda
\label{eq:coarea_formula}
\end{equation}
\end{theorem}
\begin{equation}
E_v(u) =
\int_\Omega \abs{u - v}^p
- + \beta \int_\mathbb{R} P(u^\lambda,\Omega) \, d\lambda,
+ + \beta \int_\mathbb{R} P(u \leq \lambda,\Omega) \, d\lambda.
\label{eq:energy_perimeter}
\end{equation}
-which is a good starting point for the discretization step. More
-analysis of the continuous problem is given in \cite{caselles2011total},
-together with different numerical methods for solving it. Results on how
-the set of jumps in the resulting image $u$ is contained in the set of
-jumps in the original image $v$ are also presented.
+This will be the starting point for our discretization, and it will
+allow us to decompose the energy into a sum over the discrete set of
+level values.
+
+More analysis of the continuous problem is given in
+\cite{caselles2011total}, together with different numerical methods for
+solving it. Results on how the set of jumps in the resulting image $u$
+is contained in the set of jumps in the original image $v$ are also
+presented there.
\subsection{Discrete formulation}
-Since digital images are given on a discrete grid, with values taken
-from a discrete and finite set of levels, we want to discretize
+Since digital images are given on a discrete grid with values taken from
+a discrete and finite set of levels, we want to discretize
\eqref{eq:energy_perimeter}, and solve the resulting discrete problem.
-We denote $S$ for the grid of pixels, and assume that the value of each
-pixel is taken from the set $\mathcal{L} = \{0, \hdots, L-1\}$. This is
-a reasonable assumption for grayscale images.
+We denote $\mathcal{G}$ for the grid of pixels, and assume that the
+value of each pixel is taken from the set $\mathcal{L} = \{0, \hdots,
+L-1\}$. This is a reasonable assumption for grayscale images.
\subsubsection{Regularization term}
Chambolle discretizes the total variation as
This straight-forward way of discretizing the gradient offers, according
to Chambolle, a good compromize between isotropy and stability. However,
it does not decompose into a sum over the levels of the image, which we
-need for our graph composition.
+need for our graph construction.
\begin{figure}
\input{fig/img_decomp}
\end{figure}
Following the notation used in \cite{darbon2006image}, we let $u_x$
-denote the value of the image $u$ at position $x \in S$. With a finite
-set of pixel values $\mathcal{L}$, we also have a finite number of level
-sets $u^\lambda$. Figure \ref{fig:img_decomp} visualizes
-a possible image and its decomposition into thresholded images, one for
-each level value.
-\fixme{Position $x$ already used in stats section. Problem?}
+denote the value of the image $u$ at position $x \in \mathcal{G}$. With
+a finite set of pixel values $\mathcal{L}$, we also have a finite number
+of thresholded images $u^\lambda$. Figure \ref{fig:img_decomp}
+visualizes a possible image and its decomposition into thresholded
+images, one for each level value.
\begin{figure}
\centering
level values
\begin{equation}
\TV(u)
- = \sum_{\lambda = 0}^{L-2} P(u^\lambda, S)
- = \sum_{\lambda = 0}^{L-2} \sum_{(x,y)} w_{xy}
+ = \sum_{\lambda = 0}^{L-2} P(u \leq \lambda, \Omega)
+ \approx \sum_{\lambda = 0}^{L-2} \sum_{(x,y)} w_{xy}
\abs{u_x^\lambda - u_y^\lambda}.
\label{eq:tv_discrete_int}
\end{equation}
The sum over $(x,y)$ signifies a sum over all pixels cliques of size
two, i.e.\ all pairs $(x, y)$ such that $x$ and $y$ are in a
-neighborhood relation. The $w_{xy}$ is a weight parameter. The sum over
-$\lambda$ ends at $L-2$ since $u^{L-1}$ is equal to 1 in every pixel of
-the image.
+neighborhood relation. The $w_{xy}$ is a weight parameter, and the sum
+over $\lambda$ ends at $L-2$ since $u^{L-1}$ is equal to 1 in every
+pixel of the image.
Intuitively, the perimeter of the level set $u^\lambda$ is proportional
to the number of pixels at the boundary of the set. This is again
the discrete perimeter in \eqref{eq:tv_discrete_int} converges to the
continuous perimeter in \eqref{eq:coarea_formula}. Here, $h$ is the grid
size and $d_{xy}$ is the euclidean distance of the edge from $x$ to $y$.
-The $\Delta \phi_{xy}$ is the difference between the angle of this edge
-and the next edge, if the edges are sorted by increasing angles. These
-parameters are also shown in Figure \ref{fig:common_neighborhoods}.
-Boykov and Kolmogorov prove we have convergence when all of $h$, $\Delta
-\phi_{xy}$, and $d_{xy}$ go to zero.
-
-\fixme{IS THIS EVEN NECESSARY? For two binary variables $a$ and $b$ we
- can easily verify that $\abs{a - b} = a + b - 2 a b$. Using this we
- rewrite \eqref{eq:tv_discrete_int} to
- \begin{equation}
- \TV(u) = \sum_{\lambda = 0}^{L-2} \sum_{(x,y)} w_{xy}
- \left(
- \left(1 - 2 u_y^\lambda \right) u_x^\lambda + u_y^\lambda
- \right).
- \label{eq:tv_discrete}
- \end{equation}
-}
+The number $\Delta \phi_{xy}$ is the difference between the angle of
+this edge and the next edge, if the edges are sorted by increasing
+angles. These parameters are also shown in Figure
+\ref{fig:common_neighborhoods}. Boykov and Kolmogorov prove we have
+convergence when all of $h$, $\Delta \phi_{xy}$, and $d_{xy}$ go to
+zero.
\subsubsection{Fidelity term}
Now that the total variation is discretized, we need to take care of the
\end{equation}
This allows us to discretize $\int_\Omega \abs{u - v}^p$ of
\eqref{eq:energy_perimeter} in the following way
-\fixme{Give this a name?}
\begin{equation}
\sum_x \abs{u_x - v_x}^p = \sum_x N_x(u_x)
\label{eq:norm_discrete_int}.
(1 - u^\lambda_x) + N_x(0)
\end{equation}
-\fixme{$E_\lambda^x$}
-
\subsubsection{Total energy}
\label{sec:total_energy}
are left with
\begin{equation}
E_v(u) =
- \sum_{\lambda=0}^{L-2} \sum_x E^x(u^\lambda_x)
+ \sum_{\lambda=0}^{L-2} \sum_x E^x_\lambda(u^\lambda_x)
+ \beta \sum_{\lambda = 0}^{L-2} \sum_{(x, y)} E^{x,y}(u^\lambda_x,
u^\lambda_y)
= \sum_{\lambda=0}^{L-2} F_\lambda(u^\lambda)
\end{equation}
where
\begin{align}
- E^x(u^\lambda_x) &=
+ E^x_\lambda(u^\lambda_x) &=
\big(
N_x(\lambda + 1) -
N_x(\lambda)
these obtained level sets $u^\lambda$ can be combined to make an output
image. The level sets were defined as $u^\lambda = \mathbbm{1}_{u \leq
\lambda}$, so we need them to be monotonically increasing in increasing
-level values, i.e.\
+level values, i.e.\,
\begin{equation}
u^\lambda_x \leq u^\mu_x \quad \forall \lambda \leq \mu,
\quad \forall x \in S.
\end{equation}
In the following sections we will present a graph cut algorithm that
-finds thresholded images minimizing each level, \emph{while}
-guaranteeing that they meet this requirement.
+finds thresholded images minimizing each level, \emph{while guaranteeing
+that they meet this requirement.}
\fixme{Consistently call them the fidelity and regularization terms.}
projects / prosjektoppgave / commitdiff