\section{Graph cut formulation}
-Short introduction to graphs. Notation. What is a cut, what is a minimal
-s-t-cut. How do we get from the discrete energy to the graph and why
-does finding the graph cut result in minimizing the energy of the
-current label. Boykov and Kolmogorov.
-
-\subsection{Min-cut max-flow theorem}
-Want to find the minimal s-t-cut, so we introduce flow here maybe? Or
-earlier? Finding the max flow results in also finding the minimal cut,
-and there are «good» algorithms for finding the maximal flow.
-
-\subsection{Maximum flow algorithms}
-Two different ideas. One is to always maintain a valid flow in the
-network, Ford Foulkerson, always looking for new paths from s to t along
-which we can send additional flow. This leads to Edmonds-Karp, and
-Dinic/Dinitz. Second idea is push-relabel, allowing positive excess in
-the nodes. We lift the source, and let as much flow as possible flow
-towards the sink.
+\fixme{Short introduction to graphs. Notation. What is a cut, what is a
+ minimal s-t-cut. How do we get from the discrete energy to the graph
+ and why does finding the graph cut result in minimizing the energy
+ of the current label. Boykov and Kolmogorov.}
+
+\subsection{Flow networks}
+Using the notation of \cite{cormen2009introduction} we will denote a
+directed graph as $G = (V, E)$ where $V$ is a finite set of vertices, and
+$E$ is a binary relation on $V$. We say that if $(u, v) \in E$ there is
+an edge from $u$ to $v$ in the graph $G$. When dealing with flow in a
+graph we introduce the non-negative capacity function $c : V \times V
+\to [0, \infty)$. A positive capacity $c(u, v) = q > 0$ means
+that it is possible to send between 0 and $q$ flow from $u$ to $v$.
+
+There are two special nodes in the network, the source $s$ and the sink
+$t$. In the context of maximal flow, the source can produce and the sink
+can receive, an unlimited amount of flow.
+
+For convenience we will let $c(u, v) = 0$ for any edge $(u, v) \not\in
+E$, and we do not allow self-loops in our graph. Contrary to Cormen et.
+al. in \cite{cormen2009introduction} we allow antiparallel edges in our
+graph \fixme{and it is ok.}
+
+A flow in the network $G$ is a real-valued function $f : V \times V \to
+\mathbb{R}$, such that $f(u, v)$ indicates how much flow is sent over
+the edge $(u, v)$. The flow must satisfy these two constraints
+\begin{description}
+ \item[Capacity constraint:] For all $u, v \in V, f(u, v) \leq c(u,
+ v)$, i.e., for every edge, the flow is less than or equal to the
+ capacity.
+ \item[Flow conservation:] For all $u \in V - \{s, t\}$
+ \begin{equation}
+ \sum_{v \in V} f(v, u) = \sum_{v \in V} f(u, v),
+ \end{equation}
+ i.e., for any node except the source and the sink, the
+ flow into the node must be equal to the flow out of the node.
+\end{description}
+For the capacity constraint, we purposedly allow the flow to be
+negative, and we will see why later. Finding the maximal flow that is
+possible to send from the source to the sink in a network is a much
+studied problem, which has numerous applications, and also good
+algorithms.
+
+When further discussing the max-flow problem we will need the notion of
+a residual network $G_f$, which is derived from the original network $G$
+and contains the edges along which it is possible to send additional
+flow. This means that $G_f$ contains the edges $(u, v)$ from $G$ where
+$f(u,v) < c(u,v)$. A very important realisation is that it is also
+possible to push flow \emph{back} along an edge where the flow is
+already positive, meaning that if an edge $(u, v)$ has a positive flow
+$f(u, v) > 0$, the residual graph $G_f$ should contain an edge $(v, u)$
+with capacity $f(u, v)$.
+
+Instead of constructing $G_f$ explicitly, we employ a common trick when
+modifying the flow of an edge. For every edge $(u, v)$ in the graph, we
+will also include an edge $(v, u)$, initially with a capacity of zero,
+unless otherwise specified. If the flow along $(u, v)$ is increased
+with an amount $q$, we will at the same time subtract $q$ from the flow
+of $(v, u)$. This means that $f(v, u) + q \leq c(v, u)$, so it will be
+possible to at some later stage push at least $q$ flow back from $v$ to
+$u$. The edge $(v, u)$ will be included in the residual graph $G_f$
+since it includes all edges where the flow is less than the capacity.
+
+\fixme{NOT VERY CLEAR}
+
+\fixme{$\abs{f}$ needs to be defined somewhere?}
+
+What we seek in our graph, is a the minimal $s$-$t$-cut, a
+«minimal» line through the graph that cuts a set of edges and separates
+the source from the sink.
+\begin{definition}[$s$-$t$-cut]
+ Given a network $G = (E, V, c)$, a cut $(S, T)$ of a network $G =
+ (V, E, c)$ is a partition of $V$ into $S$ and $T = V - S$ such that
+ $s \in S$ and $t \in T$. The capacity of the cut is
+ \begin{equation}
+ c(S, T) = \sum_{u \in S} \sum_{v \in S} c(u, v),
+ \end{equation}
+ and the minimum $s$-$t$-cut is the cut whose capacity is minimum
+ over all possible $s$-$t$-cuts in the network.
+\end{definition}
+When finding this minimum cut, we make use of an important duality
+theorem in network flow theory, stating that the capacity of the minimum
+$s$-$t$-cut in a network, is equal to the maximum flow from $s$ to $t$.
+\begin{theorem}[Max-flow min-cut theorem]
+ If $f$ is a flow in a network $G = (V, E, c)$ with source $s$ and
+ sink $t$, then the following is equivalent:
+ \begin{enumerate}
+ \item $f$ is a maximum flow in $G$.
+ \item The residual network $G_f$ contains no augmenting paths.
+ \item $\abs{f} = c(S, T)$ for some cut $(S, T)$ of $G$.
+ \end{enumerate}
+\end{theorem}
+See \cite{cormen2009introduction} for a proof. Note that for any
+$s$-$t$-cut $C = \{S, T\}$ in a graph $G$ with flow $\abs{f}$ from source to sink,
+the \emph{flow} across the cut is always equal to $\abs{f}$, while the
+\emph{capacity} of the cut will depend on the cut, and will only be
+equal to $\abs{f}$ for one of the minimal $s$-$t$-cuts in $G$.
+
+\fixme{But how do we find THE minimum cut? Which one do we want? The
+maximum one? The minimum one? EEEEK}
+
+\subsection{Augmenting flow algorithms}
+\begin{algorithm}
+\begin{algorithmic}
+ \Function{Ford-Foulkerson}{$G$, $s$, $t$}
+ \While{there exists a path $p$ from $s$ to $t$ in the residual
+ network $G_f$}
+ \State $f \gets \min\{res(u, v) : (u, v) \in p \}$
+ \ForAll{$(u, v) \in p$}
+ \State $flow(u, v) \mathrel{+}= f$
+ \State $flow(v, u) \mathrel{-}= f$
+ \EndFor
+ \EndWhile
+ \EndFunction
+\end{algorithmic}
+\caption{\sf The Ford-Foulkerson max-flow algorithm}
+\end{algorithm}
+
+\fixme{Use res or $c_f$???}
+
+\subsection{Graph representable energy functions}
+Kolmogorov and Zabih define in \cite{kolmogorov2002energy} the class of
+graph representable functions as follows
+\begin{definition}[Graph representable functions]
+ Let $E(x_1, \hdots, x_n)$ be a function of $n$ binary variables.
+ $E$ is graph-representable
+ if there exists a graph $G = (V, E, c)$ with terminals $s$ and $t$,
+ and a subset of vertices $V_0 = \{v_1, \hdots, v_n\} \subseteq V -
+ \{s, t\}$ such that, for any configuration $(x_1, \hdots, x_n) \in
+ \{0, 1\}^n$, the value of the energy $E(x_1, \hdots, x_n)$ is equal
+ to a constant plus the cost of the minimum $s$-$t$-cut among all
+ cuts $C = \{S, T\}$ where $x_i = 0 \Leftrightarrow v_i \in S$ and
+ $x_i = 1 \Leftrightarrow v_i \in T$, $\forall\, 1 \leq i \leq n$.
+\end{definition}
+From this definition we see that if we have a graph-representable
+function $E$ it is possible to find the exact global minimum of $E$ by
+finding the minimal $s$-$t$-cut in the corresponding graph.
+
+The following theorem if proved by Kolmogorov and Zabih in
+\cite{kolmogorov2002energy} and will be crucial in our graph
+construction.
+\begin{theorem}[Additivity]
+ The sum of a finite number of network-representable functions $G^k =
+ (V^k, E^k, c^k)$, is network-representable by $G = (E, V, c)$ where
+ $V = \cup_k V^k$ and $E = \cup_k E^k$ and the capacities $c(u,v)
+ = \sum_k c^k(u,v)$.
+\end{theorem}
+It will allow us to construct simple graphs which represents the
+different summands of \fixme{(?????)} and add them together to create
+our final graph.
+
+\fixme{Graph is usually called network when we are talking about flow.}
\subsubsection{Dinic's algorithm}
The «best» of the augmenting-path algorithms (citation needed). Makes a
\Function{Push}{$u$}
\ForAll{$v$ neighbour of $u$}
\If{$res(u, v) > 0$}
- \State $f \gets min(res(u, v), excess[u])$
+ \State $f \gets \min(res(u, v), excess[u])$
\State $flow(u, v) \mathrel{+}= f$
\State $flow(v, u) \mathrel{-}= f$
\State $excess[u] \mathrel{-}= f$
If the noise term $p(u \mid v)$ is modeled to be independently and
identically distributed Gaussian noise on each pixel, we have
\begin{equation}
- p(u \mid v) \propto \exp \left( \frac{(u - v)^2}{2\sigma^2} \right)
+ p(u \mid v) \propto
+ \prod_x \exp \left(-\frac{(u_x - v_x)^2}{2\sigma^2} \right).
\end{equation}
+Taking the logarithm we obtain
+\begin{equation}
+ \min_u \left( \sum_x \abs{u_x - v_x}^2 - \beta \ln p(u) \right).
+ \label{eq:minimizing_probability}
+\end{equation}
+If the noise had been modeled with the Laplace distribution
+\begin{equation}
+ p(u \mid v) \propto
+ \prod_x \exp \left( - \frac{\abs{u_x - v_x}}{b} \right)
+\end{equation}
+we would have obtained the $L^1$-norm instead of the $L^2$-norm in
+\eqref{eq:minimizing_probability}.
-\subsection{Functional analysis}
+\subsection{Functional analysis foundation}
\fixme{Meh heading.}
The term $\TV(u)$ is the total variation of the image, and is defined as
\TV(u)
= \int_\Omega \abs{Du}
= \sup \left\{ \int_\Omega u \,\, \mathrm{div} \, \varphi :
- \varphi \in C^\infty_c\left(\Omega, \mathbb{R}^N\right),
+ \varphi \in C^\infty_c\left(\Omega, \mathbb{R}^2\right),
\norm{\varphi}_{L^\infty(\Omega)} \leq 1
\right\}.
\end{equation}
The test functions $\varphi$ are taken from
$C^\infty_c\left(\Omega, \mathbb{R}^N\right)$, the space of smooth
functions from $\Omega$ to $\mathbb{R}^N$ with compact support.
+ \fixme{$\Omega$ open, bounded, Lipschitz boundary?}.
\end{definition}
Minimizing the total variationn will smooth out differences in the
close to the original, but more regular. The parameter $\beta$ controls
how strongly we want to regularize the image.
-Following the notation used in \cite{darbon2006image}, we let $u_s$
-denote the value of the image $u$ at position $s \in \Omega$ and
-introduce the level sets $u^\lambda$ of the image
-\begin{equation}
- u^\lambda = \idfun_{u \leq \lambda}
- \label{eq:level_set}
-\end{equation}
-which will be important when dividing the problem into smaller
-subproblems.
-
When discretizing the energy function \eqref{eq:energy_function} later
on, we want to decompose it as a sum over the different levels, or pixel
values, of the image. This is why we present the coarea formula here,
range level values $\lambda$. The coarea formula and further references
can be found in \cite{caselles2011total}.
-But first we need to introduce the space of functions with bounded
+First we need to introduce the space of functions with bounded
variation and the perimeter of a set.
\begin{definition}[Functions of bounded variation]
- The space of functions with bounded variation $\BV(\Omega)$ is the
- space of functions $u \in L^1(\Omega)$ such that the total variation
- is finite, i.e.
+ The space of functions of bounded variation $\BV(\Omega)$ is the
+ space of functions $u \in L^1(\Omega)$ for which the total variation
+ is finite, i.e.,
\begin{equation}
\BV(\Omega) = \left\{ u \in L^1(\Omega) : \TV(u) < \infty
\right\}.
from a discrete and finite set of levels, we want to discretize
\eqref{eq:energy_perimeter}, and solve the resulting discrete problem.
We assume that the image is given on a discrete grid $S$,
-where the value of each pixel is taken from the set $\mathcal{L} = [0,
-\hdots, L-1]$. A reasonable assumption for grayscale images.
+where the value of each pixel is taken from the set $\mathcal{L} = \{0,
+\hdots, L-1\}$. A reasonable assumption for grayscale images.
+
+Following the notation used in \cite{darbon2006image}, we let $u_x$
+denote the value of the image $u$ at position $x \in \Omega$ and
+introduce the level sets $u^\lambda$ of the image
+\begin{equation}
+ u^\lambda = \idfun_{u \leq \lambda}
+ \label{eq:level_set}
+\end{equation}
+which will be important when dividing the problem into smaller
+subproblems. \fixme{Position $x$ already used in stats section.}
\begin{figure}
\centering
\centering
\begin{subfigure}[b]{0.4\textwidth}
\centering
- \begin{tikzpicture}[->,>=stealth',shorten >=1pt,auto,node distance=1.5cm,
- thick,main node/.style={circle,draw}]
+ \begin{tikzpicture}[->,thick,font=\footnotesize,
+ main node/.style={circle,draw,minimum size=.7cm}]
- \node[main node] (1) {u};
- \node[main node] (2) [below of=1] {};
- \node[main node] (3) [right of=1] {v};
- \node[main node] (4) [left of=1] {};
- \node[main node] (5) [above of=1] {};
+ \node[main node] (x) at (0, 0) {x};
+ \node[main node] (2) at (-2, 0) {};
+ \node[main node] (3) at ( 2, 0) {y};
+ \node[main node] (4) at (0, -2) {};
+ \node[main node] (5) at (0, 2) {};
- \path[every node/.style={font=\sffamily\small}]
- (1) edge node [right] {} (2)
- (1) edge node [right] {$w_{uv}$} (3)
- (1) edge node [left] {} (4)
- (1) edge node [left] {} (5);
+ \path
+ (x) edge node {} (2)
+ (x) edge node [above] {$w_{xy}$} (3)
+ (x) edge node {} (4)
+ (x) edge node {} (5);
\end{tikzpicture}
\caption{Size four neighborhood.}
\label{fig:c4_grid}
~
\begin{subfigure}[b]{0.4\textwidth}
\centering
- \begin{tikzpicture}[->,>=stealth',shorten >=1pt,auto,node distance=1.5cm,
- thick,main node/.style={circle,draw}]
+ \begin{tikzpicture}[->,thick,font=\footnotesize,
+ main node/.style={circle,draw,minimum size=.7cm}]
- \node[main node] (1) {u};
- \node[main node] (2) [below of=1] {};
- \node[main node] (3) [right of=1] {};
- \node[main node] (4) [left of=1] {};
- \node[main node] (5) [above of=1] {};
- \node[main node] (6) [below right=1.5cm of 1] {};
- \node[main node] (7) [above right=1.5cm of 1] {};
- \node[main node] (8) [above left=1.5cm of 1] {};
- \node[main node] (9) [below left=1.5cm of 1] {};
+ \node[main node] (1) at (0, 0) {x};
+ \node[main node] (2) at (-2, 0) {};
+ \node[main node] (3) at ( 2, 0) {};
+ \node[main node] (4) at (0, -2) {};
+ \node[main node] (5) at (0, 2) {};
+ \node[main node] (6) at (-2, -2) {};
+ \node[main node] (7) at (-2, 2) {};
+ \node[main node] (8) at ( 2, 2) {y};
+ \node[main node] (9) at ( 2, -2) {};
\path[every node/.style={font=\sffamily\small}]
(1) edge node [right] {} (2)
(1) edge node [right] {} (7)
(1) edge node [left] {} (8)
(1) edge node [left] {} (9);
+
+ \draw (0,0) +(0:1cm) arc (0:45:1cm);
+ \path (0,0) ++(22.5:.75cm) node{$\alpha$};
+
\end{tikzpicture}
\caption{Size eight neighborhood.}
\label{fig:c8_grid}
\caption{Two common neighborhood stencils.}
\label{fig:common_neighborhoods}
\end{figure}
-We discretize the total variation in \eqref{eq:coarea_formula} as
-follows
+Chambolle discretizes the total variation as
+\begin{equation}
+ \TV(u) = \sum_{i,j} \sqrt{
+ \left( u_{i+1,j} - u_{i,j} \right)^2 +
+ \left( u_{i,j+1} - u_{i,j} \right)^2
+ }
+\end{equation}
+in his much cited paper \cite{chambolle2004algorithm}, in which he
+introduces his dual approach to the minimization problem. This
+straight-forward way of discretizing the gradient offers, according to
+Chambolle, a good compromize between isotropy and stability. However, it
+does not decompose into a sum over all the different levels of the
+image, which we need for our graph composition. This is why we
+discretize the total variation in \eqref{eq:coarea_formula}
+anisotropically as follows
\begin{equation}
\TV(u)
= \sum_{\lambda = 0}^{L-2} P(u^\lambda)
- = \sum_{\lambda = 0}^{L-2} \sum_{(u,v)} w_{uv}
- \abs{u_u^\lambda - u_v^\lambda},
+ = \sum_{\lambda = 0}^{L-2} \sum_{(x,y)} w_{xy}
+ \abs{u_x^\lambda - u_y^\lambda},
\label{eq:tv_discrete_int}
\end{equation}
where the second sum is over nodes in a neighborhood relation and the
-$w_{uv}$ is the weight of this relation. The sum only goes up to the
-level $L-2$ since $u^\lambda$ is constant equal to 1 in every pixel of
-the image. Figure \ref{fig:common_neighborhoods} shows the two most used
-neighbordhood stencils, which will be discussed more thorougly in
-\fixme{Section ??}. Boykov and Kolmogorov argue in
-\cite{boykov2003computing} that if the weight is chosen as
+$w_{uv}$ is the weight of this relation. Intuitively, the perimeter of
+the level set $u^\lambda$ is proportional to the number of edges
+crossing this perimeter, as $\abs{u^\lambda_x - u^\lambda_y}$ only
+contributes to the sum when one node is 0 and the other is 1.
+
+The sum only goes up to the level $L-2$ since $u^{L-1}$ is constant
+equal to 1 in every pixel of the image. Figure
+\ref{fig:common_neighborhoods} shows the two commonly used neighbordhood
+stencils, which will be discussed more thorougly in \fixme{Section ??}.
+Boykov and Kolmogorov argue in \cite{boykov2003computing} that if the
+weight is chosen as
\begin{equation}
- w_{uv} = \frac{h^2 \cdot \Delta \phi_{uv}}{2 \cdot \abs{d_{uv}}},
+ w_{xy} = \frac{h^2 \cdot \Delta \phi_{xy}}{2 \cdot \abs{d_{xy}}},
\end{equation}
the discrete perimeter in \eqref{eq:tv_discrete_int} converges to the
continuous perimeter in \eqref{eq:coarea_formula}. Here, $h$ is the grid
-size and $d_{uv}$ is the euclidean distance of the edge. The $\Delta
-\phi_{uv}$ is the difference between the angle of this edge and the
+size and $d_{xy}$ is the euclidean distance of the edge. The $\Delta
+\phi_{xy}$ is the difference between the angle of this edge and the
next edge, if the edges are sorted by increasing angles. These
parameters are also shown in Figure \ref{fig:common_neighborhoods}.
Boykov and Kolmogorov prove we have convergence when all of $h$, $\Delta
-\phi_{uv}$, and $d_{uv}$ go to zero.
+\phi_{xy}$, and $d_{xy}$ go to zero. In other words, if the grid size
+decreases and we at the same time increase the density of the
+neighborhood.
\fixme{Maybe move (and expand) this somewhere else? Where the grids are
discussed for example? Also, the section is a bit messy.}
For two binary variables $a$ and $b$ we can easily verify that $\abs{a -
b} = a + b - 2 a b$. Using this we rewrite \eqref{eq:tv_discrete_int} to
\begin{equation}
- \TV(u) = \sum_{\lambda = 0}^{L-2} \sum_{(u,v)} w_{uv}
+ \TV(u) = \sum_{\lambda = 0}^{L-2} \sum_{(x,y)} w_{xy}
\left(
- \left(1 - 2 u_t^\lambda \right) u_s^\lambda + u_t^\lambda
+ \left(1 - 2 u_y^\lambda \right) u_x^\lambda + u_y^\lambda
\right).
\label{eq:tv_discrete}
\end{equation}
-We define the following function for some pixel value $x$ and some pixel
-position $u$ in the original image $v$, which is the value of the energy
-if we were to color pixel $u$ with label $x$
+We define the following function for some pixel value $k$ and some pixel
+position $x$ in the original image $v$, which is the value of the energy
+if we were to color pixel $u$ with label $k$
\begin{equation}
- N_u(x) = \abs{x - v_u}^p.
+ N_x(k) = \abs{k - v_x}^p.
\end{equation}
This allows us to discretize $\int_\Omega \abs{u - v}^p$ of
\eqref{eq:energy_perimeter} in the following way
\fixme{Give this a name?}
\begin{equation}
- \sum_u \abs{u_u - v_u}^p = \sum_u N_u(u_u)
+ \sum_x \abs{u_x - v_x}^p = \sum_x N_x(u_x)
\label{eq:norm_discrete_int}.
\end{equation}
We want to write the energy as a sum over the different levels of the
\end{equation}
Using this we rewrite \eqref{eq:norm_discrete_int} and obtain
\begin{equation}
- \sum_{\lambda=0}^{L-2} \sum_u
+ \sum_{\lambda=0}^{L-2} \sum_x
\big(
- N_u(\lambda + 1) -
- N_u(\lambda)
+ N_x(\lambda + 1) -
+ N_x(\lambda)
\big)
- (1 - u^\lambda) + N_u(0)
+ (1 - u^\lambda_x) + N_x(0)
\end{equation}
We have now discretized the energy function and decomposed it into a sum
over all the levels $\lambda$ of the image. If we minimize each level
\mathbbm{1}_{u \leq \lambda}$, so we need them to be monotonely
increasing in increasing level values, i.e.\
\begin{equation}
- u^\lambda_u \leq u^\mu_u \quad \forall \lambda \leq \mu,
- \quad \forall u \in S.
+ u^\lambda_x \leq u^\mu_x \quad \forall \lambda \leq \mu,
+ \quad \forall x \in S.
\end{equation}
\fixme{Want to use $u$ and $v$ as pixels/nodes, what to use as image?
projects / prosjektoppgave / commitdiff