From: Bjørn Rustad Date: Wed, 19 Mar 2014 08:02:08 +0000 (+0100) Subject: Mer om grafer X-Git-Url: http://git.rustad.me/?a=commitdiff_plain;h=9bbf058fc2fc1f2428799280acd59774fb1a8761;p=prosjektoppgave Mer om grafer --- diff --git a/bib.bib b/bib.bib index e500e57..571b04e 100644 --- a/bib.bib +++ b/bib.bib @@ -129,4 +129,32 @@ publisher={Teubner Stuttgart} } +@article{chambolle2004algorithm, + title={An algorithm for total variation minimization and applications}, + author={Chambolle, Antonin}, + journal={Journal of Mathematical imaging and vision}, + volume={20}, + number={1-2}, + pages={89--97}, + year={2004}, + publisher={Springer} +} + +@incollection{kolmogorov2002energy, + title={What energy functions can be minimized via graph cuts?}, + author={Kolmogorov, Vladimir and Zabih, Ramin}, + booktitle={Computer Vision—ECCV 2002}, + pages={65--81}, + year={2002}, + publisher={Springer} +} + +@book{cormen2009introduction, + title={Introduction to algorithms}, + author={Cormen, Thomas H. and Leiserson, Charles E. and Rivest, Ronald L. and Stein, Clifford}, + edition={Third}, + year={2009}, + publisher={MIT press} +} + diff --git a/cut.tex b/cut.tex index da8f62b..9f23628 100644 --- a/cut.tex +++ b/cut.tex @@ -1,21 +1,156 @@ \section{Graph cut formulation} -Short introduction to graphs. Notation. What is a cut, what is a minimal -s-t-cut. How do we get from the discrete energy to the graph and why -does finding the graph cut result in minimizing the energy of the -current label. Boykov and Kolmogorov. - -\subsection{Min-cut max-flow theorem} -Want to find the minimal s-t-cut, so we introduce flow here maybe? Or -earlier? Finding the max flow results in also finding the minimal cut, -and there are «good» algorithms for finding the maximal flow. - -\subsection{Maximum flow algorithms} -Two different ideas. One is to always maintain a valid flow in the -network, Ford Foulkerson, always looking for new paths from s to t along -which we can send additional flow. This leads to Edmonds-Karp, and -Dinic/Dinitz. Second idea is push-relabel, allowing positive excess in -the nodes. We lift the source, and let as much flow as possible flow -towards the sink. +\fixme{Short introduction to graphs. Notation. What is a cut, what is a + minimal s-t-cut. How do we get from the discrete energy to the graph + and why does finding the graph cut result in minimizing the energy + of the current label. Boykov and Kolmogorov.} + +\subsection{Flow networks} +Using the notation of \cite{cormen2009introduction} we will denote a +directed graph as $G = (V, E)$ where $V$ is a finite set of vertices, and +$E$ is a binary relation on $V$. We say that if $(u, v) \in E$ there is +an edge from $u$ to $v$ in the graph $G$. When dealing with flow in a +graph we introduce the non-negative capacity function $c : V \times V +\to [0, \infty)$. A positive capacity $c(u, v) = q > 0$ means +that it is possible to send between 0 and $q$ flow from $u$ to $v$. + +There are two special nodes in the network, the source $s$ and the sink +$t$. In the context of maximal flow, the source can produce and the sink +can receive, an unlimited amount of flow. + +For convenience we will let $c(u, v) = 0$ for any edge $(u, v) \not\in +E$, and we do not allow self-loops in our graph. Contrary to Cormen et. +al. in \cite{cormen2009introduction} we allow antiparallel edges in our +graph \fixme{and it is ok.} + +A flow in the network $G$ is a real-valued function $f : V \times V \to +\mathbb{R}$, such that $f(u, v)$ indicates how much flow is sent over +the edge $(u, v)$. The flow must satisfy these two constraints +\begin{description} + \item[Capacity constraint:] For all $u, v \in V, f(u, v) \leq c(u, + v)$, i.e., for every edge, the flow is less than or equal to the + capacity. + \item[Flow conservation:] For all $u \in V - \{s, t\}$ + \begin{equation} + \sum_{v \in V} f(v, u) = \sum_{v \in V} f(u, v), + \end{equation} + i.e., for any node except the source and the sink, the + flow into the node must be equal to the flow out of the node. +\end{description} +For the capacity constraint, we purposedly allow the flow to be +negative, and we will see why later. Finding the maximal flow that is +possible to send from the source to the sink in a network is a much +studied problem, which has numerous applications, and also good +algorithms. + +When further discussing the max-flow problem we will need the notion of +a residual network $G_f$, which is derived from the original network $G$ +and contains the edges along which it is possible to send additional +flow. This means that $G_f$ contains the edges $(u, v)$ from $G$ where +$f(u,v) < c(u,v)$. A very important realisation is that it is also +possible to push flow \emph{back} along an edge where the flow is +already positive, meaning that if an edge $(u, v)$ has a positive flow +$f(u, v) > 0$, the residual graph $G_f$ should contain an edge $(v, u)$ +with capacity $f(u, v)$. + +Instead of constructing $G_f$ explicitly, we employ a common trick when +modifying the flow of an edge. For every edge $(u, v)$ in the graph, we +will also include an edge $(v, u)$, initially with a capacity of zero, +unless otherwise specified. If the flow along $(u, v)$ is increased +with an amount $q$, we will at the same time subtract $q$ from the flow +of $(v, u)$. This means that $f(v, u) + q \leq c(v, u)$, so it will be +possible to at some later stage push at least $q$ flow back from $v$ to +$u$. The edge $(v, u)$ will be included in the residual graph $G_f$ +since it includes all edges where the flow is less than the capacity. + +\fixme{NOT VERY CLEAR} + +\fixme{$\abs{f}$ needs to be defined somewhere?} + +What we seek in our graph, is a the minimal $s$-$t$-cut, a +«minimal» line through the graph that cuts a set of edges and separates +the source from the sink. +\begin{definition}[$s$-$t$-cut] + Given a network $G = (E, V, c)$, a cut $(S, T)$ of a network $G = + (V, E, c)$ is a partition of $V$ into $S$ and $T = V - S$ such that + $s \in S$ and $t \in T$. The capacity of the cut is + \begin{equation} + c(S, T) = \sum_{u \in S} \sum_{v \in S} c(u, v), + \end{equation} + and the minimum $s$-$t$-cut is the cut whose capacity is minimum + over all possible $s$-$t$-cuts in the network. +\end{definition} +When finding this minimum cut, we make use of an important duality +theorem in network flow theory, stating that the capacity of the minimum +$s$-$t$-cut in a network, is equal to the maximum flow from $s$ to $t$. +\begin{theorem}[Max-flow min-cut theorem] + If $f$ is a flow in a network $G = (V, E, c)$ with source $s$ and + sink $t$, then the following is equivalent: + \begin{enumerate} + \item $f$ is a maximum flow in $G$. + \item The residual network $G_f$ contains no augmenting paths. + \item $\abs{f} = c(S, T)$ for some cut $(S, T)$ of $G$. + \end{enumerate} +\end{theorem} +See \cite{cormen2009introduction} for a proof. Note that for any +$s$-$t$-cut $C = \{S, T\}$ in a graph $G$ with flow $\abs{f}$ from source to sink, +the \emph{flow} across the cut is always equal to $\abs{f}$, while the +\emph{capacity} of the cut will depend on the cut, and will only be +equal to $\abs{f}$ for one of the minimal $s$-$t$-cuts in $G$. + +\fixme{But how do we find THE minimum cut? Which one do we want? The +maximum one? The minimum one? EEEEK} + +\subsection{Augmenting flow algorithms} +\begin{algorithm} +\begin{algorithmic} + \Function{Ford-Foulkerson}{$G$, $s$, $t$} + \While{there exists a path $p$ from $s$ to $t$ in the residual + network $G_f$} + \State $f \gets \min\{res(u, v) : (u, v) \in p \}$ + \ForAll{$(u, v) \in p$} + \State $flow(u, v) \mathrel{+}= f$ + \State $flow(v, u) \mathrel{-}= f$ + \EndFor + \EndWhile + \EndFunction +\end{algorithmic} +\caption{\sf The Ford-Foulkerson max-flow algorithm} +\end{algorithm} + +\fixme{Use res or $c_f$???} + +\subsection{Graph representable energy functions} +Kolmogorov and Zabih define in \cite{kolmogorov2002energy} the class of +graph representable functions as follows +\begin{definition}[Graph representable functions] + Let $E(x_1, \hdots, x_n)$ be a function of $n$ binary variables. + $E$ is graph-representable + if there exists a graph $G = (V, E, c)$ with terminals $s$ and $t$, + and a subset of vertices $V_0 = \{v_1, \hdots, v_n\} \subseteq V - + \{s, t\}$ such that, for any configuration $(x_1, \hdots, x_n) \in + \{0, 1\}^n$, the value of the energy $E(x_1, \hdots, x_n)$ is equal + to a constant plus the cost of the minimum $s$-$t$-cut among all + cuts $C = \{S, T\}$ where $x_i = 0 \Leftrightarrow v_i \in S$ and + $x_i = 1 \Leftrightarrow v_i \in T$, $\forall\, 1 \leq i \leq n$. +\end{definition} +From this definition we see that if we have a graph-representable +function $E$ it is possible to find the exact global minimum of $E$ by +finding the minimal $s$-$t$-cut in the corresponding graph. + +The following theorem if proved by Kolmogorov and Zabih in +\cite{kolmogorov2002energy} and will be crucial in our graph +construction. +\begin{theorem}[Additivity] + The sum of a finite number of network-representable functions $G^k = + (V^k, E^k, c^k)$, is network-representable by $G = (E, V, c)$ where + $V = \cup_k V^k$ and $E = \cup_k E^k$ and the capacities $c(u,v) + = \sum_k c^k(u,v)$. +\end{theorem} +It will allow us to construct simple graphs which represents the +different summands of \fixme{(?????)} and add them together to create +our final graph. + +\fixme{Graph is usually called network when we are talking about flow.} \subsubsection{Dinic's algorithm} The «best» of the augmenting-path algorithms (citation needed). Makes a @@ -69,7 +204,7 @@ article the gap relabeling heuristic is also called RELABEL-GLOBAL. \Function{Push}{$u$} \ForAll{$v$ neighbour of $u$} \If{$res(u, v) > 0$} - \State $f \gets min(res(u, v), excess[u])$ + \State $f \gets \min(res(u, v), excess[u])$ \State $flow(u, v) \mathrel{+}= f$ \State $flow(v, u) \mathrel{-}= f$ \State $excess[u] \mathrel{-}= f$ diff --git a/total.tex b/total.tex index 95f9caa..0da5459 100644 --- a/total.tex +++ b/total.tex @@ -30,10 +30,23 @@ equivalent to the formulation If the noise term $p(u \mid v)$ is modeled to be independently and identically distributed Gaussian noise on each pixel, we have \begin{equation} - p(u \mid v) \propto \exp \left( \frac{(u - v)^2}{2\sigma^2} \right) + p(u \mid v) \propto + \prod_x \exp \left(-\frac{(u_x - v_x)^2}{2\sigma^2} \right). \end{equation} +Taking the logarithm we obtain +\begin{equation} + \min_u \left( \sum_x \abs{u_x - v_x}^2 - \beta \ln p(u) \right). + \label{eq:minimizing_probability} +\end{equation} +If the noise had been modeled with the Laplace distribution +\begin{equation} + p(u \mid v) \propto + \prod_x \exp \left( - \frac{\abs{u_x - v_x}}{b} \right) +\end{equation} +we would have obtained the $L^1$-norm instead of the $L^2$-norm in +\eqref{eq:minimizing_probability}. -\subsection{Functional analysis} +\subsection{Functional analysis foundation} \fixme{Meh heading.} The term $\TV(u)$ is the total variation of the image, and is defined as @@ -46,13 +59,14 @@ follows: \TV(u) = \int_\Omega \abs{Du} = \sup \left\{ \int_\Omega u \,\, \mathrm{div} \, \varphi : - \varphi \in C^\infty_c\left(\Omega, \mathbb{R}^N\right), + \varphi \in C^\infty_c\left(\Omega, \mathbb{R}^2\right), \norm{\varphi}_{L^\infty(\Omega)} \leq 1 \right\}. \end{equation} The test functions $\varphi$ are taken from $C^\infty_c\left(\Omega, \mathbb{R}^N\right)$, the space of smooth functions from $\Omega$ to $\mathbb{R}^N$ with compact support. + \fixme{$\Omega$ open, bounded, Lipschitz boundary?}. \end{definition} Minimizing the total variationn will smooth out differences in the @@ -62,16 +76,6 @@ close to the input image $v$. Combining the two will give us an image close to the original, but more regular. The parameter $\beta$ controls how strongly we want to regularize the image. -Following the notation used in \cite{darbon2006image}, we let $u_s$ -denote the value of the image $u$ at position $s \in \Omega$ and -introduce the level sets $u^\lambda$ of the image -\begin{equation} - u^\lambda = \idfun_{u \leq \lambda} - \label{eq:level_set} -\end{equation} -which will be important when dividing the problem into smaller -subproblems. - When discretizing the energy function \eqref{eq:energy_function} later on, we want to decompose it as a sum over the different levels, or pixel values, of the image. This is why we present the coarea formula here, @@ -79,12 +83,12 @@ and use it to write the total variation $\TV(u)$ as an integral over the range level values $\lambda$. The coarea formula and further references can be found in \cite{caselles2011total}. -But first we need to introduce the space of functions with bounded +First we need to introduce the space of functions with bounded variation and the perimeter of a set. \begin{definition}[Functions of bounded variation] - The space of functions with bounded variation $\BV(\Omega)$ is the - space of functions $u \in L^1(\Omega)$ such that the total variation - is finite, i.e. + The space of functions of bounded variation $\BV(\Omega)$ is the + space of functions $u \in L^1(\Omega)$ for which the total variation + is finite, i.e., \begin{equation} \BV(\Omega) = \left\{ u \in L^1(\Omega) : \TV(u) < \infty \right\}. @@ -134,8 +138,18 @@ Since digital images are given on a discrete grid, with values taken from a discrete and finite set of levels, we want to discretize \eqref{eq:energy_perimeter}, and solve the resulting discrete problem. We assume that the image is given on a discrete grid $S$, -where the value of each pixel is taken from the set $\mathcal{L} = [0, -\hdots, L-1]$. A reasonable assumption for grayscale images. +where the value of each pixel is taken from the set $\mathcal{L} = \{0, +\hdots, L-1\}$. A reasonable assumption for grayscale images. + +Following the notation used in \cite{darbon2006image}, we let $u_x$ +denote the value of the image $u$ at position $x \in \Omega$ and +introduce the level sets $u^\lambda$ of the image +\begin{equation} + u^\lambda = \idfun_{u \leq \lambda} + \label{eq:level_set} +\end{equation} +which will be important when dividing the problem into smaller +subproblems. \fixme{Position $x$ already used in stats section.} \begin{figure} \centering @@ -174,20 +188,20 @@ one-dimensional image divided into its level sets. \centering \begin{subfigure}[b]{0.4\textwidth} \centering - \begin{tikzpicture}[->,>=stealth',shorten >=1pt,auto,node distance=1.5cm, - thick,main node/.style={circle,draw}] + \begin{tikzpicture}[->,thick,font=\footnotesize, + main node/.style={circle,draw,minimum size=.7cm}] - \node[main node] (1) {u}; - \node[main node] (2) [below of=1] {}; - \node[main node] (3) [right of=1] {v}; - \node[main node] (4) [left of=1] {}; - \node[main node] (5) [above of=1] {}; + \node[main node] (x) at (0, 0) {x}; + \node[main node] (2) at (-2, 0) {}; + \node[main node] (3) at ( 2, 0) {y}; + \node[main node] (4) at (0, -2) {}; + \node[main node] (5) at (0, 2) {}; - \path[every node/.style={font=\sffamily\small}] - (1) edge node [right] {} (2) - (1) edge node [right] {$w_{uv}$} (3) - (1) edge node [left] {} (4) - (1) edge node [left] {} (5); + \path + (x) edge node {} (2) + (x) edge node [above] {$w_{xy}$} (3) + (x) edge node {} (4) + (x) edge node {} (5); \end{tikzpicture} \caption{Size four neighborhood.} \label{fig:c4_grid} @@ -195,18 +209,18 @@ one-dimensional image divided into its level sets. ~ \begin{subfigure}[b]{0.4\textwidth} \centering - \begin{tikzpicture}[->,>=stealth',shorten >=1pt,auto,node distance=1.5cm, - thick,main node/.style={circle,draw}] + \begin{tikzpicture}[->,thick,font=\footnotesize, + main node/.style={circle,draw,minimum size=.7cm}] - \node[main node] (1) {u}; - \node[main node] (2) [below of=1] {}; - \node[main node] (3) [right of=1] {}; - \node[main node] (4) [left of=1] {}; - \node[main node] (5) [above of=1] {}; - \node[main node] (6) [below right=1.5cm of 1] {}; - \node[main node] (7) [above right=1.5cm of 1] {}; - \node[main node] (8) [above left=1.5cm of 1] {}; - \node[main node] (9) [below left=1.5cm of 1] {}; + \node[main node] (1) at (0, 0) {x}; + \node[main node] (2) at (-2, 0) {}; + \node[main node] (3) at ( 2, 0) {}; + \node[main node] (4) at (0, -2) {}; + \node[main node] (5) at (0, 2) {}; + \node[main node] (6) at (-2, -2) {}; + \node[main node] (7) at (-2, 2) {}; + \node[main node] (8) at ( 2, 2) {y}; + \node[main node] (9) at ( 2, -2) {}; \path[every node/.style={font=\sffamily\small}] (1) edge node [right] {} (2) @@ -217,6 +231,10 @@ one-dimensional image divided into its level sets. (1) edge node [right] {} (7) (1) edge node [left] {} (8) (1) edge node [left] {} (9); + + \draw (0,0) +(0:1cm) arc (0:45:1cm); + \path (0,0) ++(22.5:.75cm) node{$\alpha$}; + \end{tikzpicture} \caption{Size eight neighborhood.} \label{fig:c8_grid} @@ -224,33 +242,53 @@ one-dimensional image divided into its level sets. \caption{Two common neighborhood stencils.} \label{fig:common_neighborhoods} \end{figure} -We discretize the total variation in \eqref{eq:coarea_formula} as -follows +Chambolle discretizes the total variation as +\begin{equation} + \TV(u) = \sum_{i,j} \sqrt{ + \left( u_{i+1,j} - u_{i,j} \right)^2 + + \left( u_{i,j+1} - u_{i,j} \right)^2 + } +\end{equation} +in his much cited paper \cite{chambolle2004algorithm}, in which he +introduces his dual approach to the minimization problem. This +straight-forward way of discretizing the gradient offers, according to +Chambolle, a good compromize between isotropy and stability. However, it +does not decompose into a sum over all the different levels of the +image, which we need for our graph composition. This is why we +discretize the total variation in \eqref{eq:coarea_formula} +anisotropically as follows \begin{equation} \TV(u) = \sum_{\lambda = 0}^{L-2} P(u^\lambda) - = \sum_{\lambda = 0}^{L-2} \sum_{(u,v)} w_{uv} - \abs{u_u^\lambda - u_v^\lambda}, + = \sum_{\lambda = 0}^{L-2} \sum_{(x,y)} w_{xy} + \abs{u_x^\lambda - u_y^\lambda}, \label{eq:tv_discrete_int} \end{equation} where the second sum is over nodes in a neighborhood relation and the -$w_{uv}$ is the weight of this relation. The sum only goes up to the -level $L-2$ since $u^\lambda$ is constant equal to 1 in every pixel of -the image. Figure \ref{fig:common_neighborhoods} shows the two most used -neighbordhood stencils, which will be discussed more thorougly in -\fixme{Section ??}. Boykov and Kolmogorov argue in -\cite{boykov2003computing} that if the weight is chosen as +$w_{uv}$ is the weight of this relation. Intuitively, the perimeter of +the level set $u^\lambda$ is proportional to the number of edges +crossing this perimeter, as $\abs{u^\lambda_x - u^\lambda_y}$ only +contributes to the sum when one node is 0 and the other is 1. + +The sum only goes up to the level $L-2$ since $u^{L-1}$ is constant +equal to 1 in every pixel of the image. Figure +\ref{fig:common_neighborhoods} shows the two commonly used neighbordhood +stencils, which will be discussed more thorougly in \fixme{Section ??}. +Boykov and Kolmogorov argue in \cite{boykov2003computing} that if the +weight is chosen as \begin{equation} - w_{uv} = \frac{h^2 \cdot \Delta \phi_{uv}}{2 \cdot \abs{d_{uv}}}, + w_{xy} = \frac{h^2 \cdot \Delta \phi_{xy}}{2 \cdot \abs{d_{xy}}}, \end{equation} the discrete perimeter in \eqref{eq:tv_discrete_int} converges to the continuous perimeter in \eqref{eq:coarea_formula}. Here, $h$ is the grid -size and $d_{uv}$ is the euclidean distance of the edge. The $\Delta -\phi_{uv}$ is the difference between the angle of this edge and the +size and $d_{xy}$ is the euclidean distance of the edge. The $\Delta +\phi_{xy}$ is the difference between the angle of this edge and the next edge, if the edges are sorted by increasing angles. These parameters are also shown in Figure \ref{fig:common_neighborhoods}. Boykov and Kolmogorov prove we have convergence when all of $h$, $\Delta -\phi_{uv}$, and $d_{uv}$ go to zero. +\phi_{xy}$, and $d_{xy}$ go to zero. In other words, if the grid size +decreases and we at the same time increase the density of the +neighborhood. \fixme{Maybe move (and expand) this somewhere else? Where the grids are discussed for example? Also, the section is a bit messy.} @@ -259,24 +297,24 @@ discussed for example? Also, the section is a bit messy.} For two binary variables $a$ and $b$ we can easily verify that $\abs{a - b} = a + b - 2 a b$. Using this we rewrite \eqref{eq:tv_discrete_int} to \begin{equation} - \TV(u) = \sum_{\lambda = 0}^{L-2} \sum_{(u,v)} w_{uv} + \TV(u) = \sum_{\lambda = 0}^{L-2} \sum_{(x,y)} w_{xy} \left( - \left(1 - 2 u_t^\lambda \right) u_s^\lambda + u_t^\lambda + \left(1 - 2 u_y^\lambda \right) u_x^\lambda + u_y^\lambda \right). \label{eq:tv_discrete} \end{equation} -We define the following function for some pixel value $x$ and some pixel -position $u$ in the original image $v$, which is the value of the energy -if we were to color pixel $u$ with label $x$ +We define the following function for some pixel value $k$ and some pixel +position $x$ in the original image $v$, which is the value of the energy +if we were to color pixel $u$ with label $k$ \begin{equation} - N_u(x) = \abs{x - v_u}^p. + N_x(k) = \abs{k - v_x}^p. \end{equation} This allows us to discretize $\int_\Omega \abs{u - v}^p$ of \eqref{eq:energy_perimeter} in the following way \fixme{Give this a name?} \begin{equation} - \sum_u \abs{u_u - v_u}^p = \sum_u N_u(u_u) + \sum_x \abs{u_x - v_x}^p = \sum_x N_x(u_x) \label{eq:norm_discrete_int}. \end{equation} We want to write the energy as a sum over the different levels of the @@ -297,12 +335,12 @@ function $N(k)$ taking values $k \in \mathcal{L}$ \end{equation} Using this we rewrite \eqref{eq:norm_discrete_int} and obtain \begin{equation} - \sum_{\lambda=0}^{L-2} \sum_u + \sum_{\lambda=0}^{L-2} \sum_x \big( - N_u(\lambda + 1) - - N_u(\lambda) + N_x(\lambda + 1) - + N_x(\lambda) \big) - (1 - u^\lambda) + N_u(0) + (1 - u^\lambda_x) + N_x(0) \end{equation} We have now discretized the energy function and decomposed it into a sum over all the levels $\lambda$ of the image. If we minimize each level @@ -312,8 +350,8 @@ make an output image. The level sets were defined as $u^\lambda = \mathbbm{1}_{u \leq \lambda}$, so we need them to be monotonely increasing in increasing level values, i.e.\ \begin{equation} - u^\lambda_u \leq u^\mu_u \quad \forall \lambda \leq \mu, - \quad \forall u \in S. + u^\lambda_x \leq u^\mu_x \quad \forall \lambda \leq \mu, + \quad \forall x \in S. \end{equation} \fixme{Want to use $u$ and $v$ as pixels/nodes, what to use as image?