\subsection{Fidelity term}
-The fidelity term can be discretize as in my project work
-\cite{project}. For some pixel position $x \in \mathcal{G}$ and some level value $k \in
+Since it is not affected by our introduction of the anisotropy tensor,
+the fidelity term can be discretized as in my project work
+\cite{project}. For some pixel position $x \in \mathcal{G}$ and some
+level value $k \in
\mathcal{L}$, we define the following function
\begin{equation}
N_x(k) = \abs{k - v_x}^p
of $k$. This allows us write
\begin{equation}
\int_\Omega \abs{u - v}^p\, dx \approx \sum_{x \in \mathcal{G}}
- \abs{u_x - v_x}^p = \sum_{x \in \mathcal{G}} N_x(u_x).
+ \abs{u_x - v_x}^p \Delta x = \sum_{x \in \mathcal{G}} N_x(u_x)
+ \Delta x.
\label{eq:fidelity_approx_1}
\end{equation}
The reason we introduce the function $N_x(k)$ is that we want to apply
\in \mathcal{G}}
\big( N_x(\lambda + 1) - N_x(\lambda) \big) \, u_x^\lambda + N_x(0).
\end{equation}
-Note that since our image takes values in $\mathcal{L} = \{0, \hdots,
-L-1\}$, the thresholded image $u^{L-1}$ is equal to zero everywhere.
+As our domain is discretized uniformly, we drop the constant
+$\Delta x$, and absorb it into our parameter $\beta$. Note that since
+our image takes values in $\mathcal{L} = \{0, \hdots, L-1\}$, the
+thresholded image $u^{L-1}$ is equal to zero everywhere.
\subsection{Regularization term}
\int_{-\infty}^\infty \PerA( \{ u > \lambda \}; \Omega) \, d\lambda
\approx \sum_{\lambda = 0}^{L-2} \PerA( \{ u > \lambda \}; \Omega)
\, \Delta \lambda.
+ \label{eq:per_approx1}
\end{equation}
Note that we will later ignore the $\Delta \lambda$ difference, as we can
just absorb it into the $\beta$ parameter of
\end{equation}
The set of lines $\mathcal{L}$ has been discretized to the lines
$\mathcal{L}_D$. Note that we are approximating the length of the
-\emph{differentiable} curve $C$ in $\Omega$.
-
-Since the final goal is to work with digital images, it makes sense to
-discretize our domain $\Omega$ as a regular grid $\mathcal{G}$. Our
-image is then reduced to a function $u : \mathcal{G} \to \mathcal{L}$.
-\fixme{mathcal L is now two things.} Moreover, the level sets $\{ u >
-\lambda\}$ will be functions taking the value of 0 or 1 on this grid, as
-shown in Figure \fixme{ref}.
+\emph{differentiable} curve $C$ in $\Omega$. Being a difference in the
+$\rho$ parameter of our line discretization in Figure
+\ref{fig:line_param}, the difference $\Delta \rho$ represents the
+distance from one line to the next in a line family as shown in Figure
+\ref{fig:line_family}. The difference $\Delta \phi$ is taken to be the
+average of the distance to the two neighboring line families as shown in
+Figure \ref{fig:line_neigh}
+
+In the discrete setting our domain $\Omega$ is discretized as a regular
+grid $\mathcal{G}$. Our image is then reduced to a function $u :
+\mathcal{G} \to \mathcal{L}$. \fixme{mathcal L is now two things.}
+Moreover, the level sets $\{ u > \lambda\}$ will be functions taking the
+value of 0 or 1 on this grid, as shown in Figure \fixme{ref}.
The choice of our discrete set of lines $\mathcal{L}_D$ is important, as
it will decide the accuracy of our approximation in
-\eqref{eq:cauchy_crofton_approx1}. We will only consider lines going
-through at least two points in our grid $\mathcal{G}$, and for now we will
-consider a discretization which is uniform throughout the domain,
-meaning that $\Delta \rho$ is constant for each line family, and that in
-each grid point, there is a line from each family. \fixme{moar
-explanation.} The
-set of lines can then be represented by the neighborhood of a pixel as
+\eqref{eq:cauchy_crofton_approx1}. We need some sensible restrictions on
+the set $\mathcal{L}_D$ to simplify the further discussion. All lines
+intersect at least two grid points, and from the periodicity of our grid
+they thus intersect an infinite number of grid points. This puts some
+restrictions on the angles we can choose. For each angle, we include all
+possible lines of that family, meaning there are no grid points without
+a line of that family intersecting it.
+
+The set of lines can then be represented by the neighborhood of a pixel as
shown in Figure \ref{fig:line_neigh}. Extending the edges shown in the
figure gives all lines going through the point considered. Figure
\ref{fig:line_family} shows all lines of a given family, i.e.\ lines
calculations of these points will not fit into our graph cut framework
later, and thus for an edge $e$ we will consider only the question of
``did $e$ cross $C$ or not?'' This amounts to checking whether the
-terminals of $e$ lie on each side of the curve $C$, and the
+terminals of $e$ lie on different sides of the curve $C$, and the
approximation is exact for zero or one intersection points, but will, as
we see in Figure \ref{fig:curve_edge}, not be entirely correct when we
have more.
\label{eq:tensor_approx}
\end{equation}
the component-wise average of the tensors in the two end points of the
-edge. \fixme{really? componentwise? will that not mess up the
-eigenvalues? sure, a bit, but it won't change consistency..}
+edge. Recall that we have already done some spatial smoothing of the
+structure tensor in \eqref{eq:s_def} corresponding to the
+\emph{integration scale} $\rho$, and thus we expect the tensors $M(a)$
+and $M(b)$ to be similar for edges $e$ of reasonably short length.
-\fixme{
- we must define what we mean by a reasonable line family. meaning
- each line goes through more than one grid point. and there are no
- grid points without a line through it
-}
+\fixme{really? componentwise? will that not mess up the
+eigenvalues? sure, a bit, but it won't change consistency..}
\begin{figure}
\input{fig/area_proof}
\end{figure}
+We have now almost arrived at our final curve length approximation, but
+we need a way to calculate the inter-line distance $\Delta \rho$ which
+will be provided by the following lemma.
\begin{lemma}
For each family of lines given by an angle parameter $\phi$ in the
regular grid of size $\delta$ we have the relation
\Delta \rho &= \min_{(p\prime, q\prime)} \left\{
\left\langle \delta [p - p\prime, q - q\prime],
\frac{e^\perp}{\norm{e^\perp}} \right\rangle \right\} \\
- &= \min \left\{\delta^2 \cdot \frac{t(p-p\prime) - s(q -
- q\prime)}{\norm{e}} \right\}.
+ &= \min_{(p\prime, q\prime)} \left\{\delta^2 \cdot
+ \frac{t(p-p\prime) - s(q - q\prime)}{\norm{e}} \right\}.
\end{aligned}
\end{equation}
Since $s$ and $t$ are coprime, there exists $a, b \in \mathbb{Z}$
\Delta \rho = \frac{\delta^2}{\norm{e}}.
\end{equation}
\end{proof}
-Inserting
-this \fixme{and the tensor approx} into the curve length approximation
-of \fixme{ref} we obtain
+Inserting $\Delta \rho = \delta^2 / \norm{e}$ and the tensor
+approximation of \eqref{eq:tensor_approx} into the curve length
+approximation of \eqref{eq:cauchy_crofton_approx1} we obtain
\begin{equation}
\abs{C}_M \approx \sum_{e \cap C} \frac{\det M(e) \norm{e}^2
\, \delta^2 \, \Delta\phi}{2 \left(e^T \cdot M(e) \cdot
e\right)^{\sfrac{3}{2}}},
+ \label{eq:cauchy_crofton_approx2}
\end{equation}
where the sum is over all edges crossing the curve an odd number of
times.
-Going back to the perimeter of the level set $\{ u > \lambda \}$ we can
-rewrite the sum to be a sum over all edges that goes from one side of
-the set to the other such that
+
+The curve length we initially wanted to calculate was the perimeter
+$\PerA(\{u > \lambda\}; \Omega)$ in \eqref{eq:per_approx1}. As this
+curve is closed, we know that every edge intersecting it must have one
+terminal in $\{ u > \lambda \}$ and the other outside. Thus we rewrite
+the sum over $e \cap C$ such that
\begin{equation}
- \PerA(\{u > \lambda\}; \mathcal{G}) = \sum_{e_{ab}} \abs{
+ \PerA(\{u > \lambda\}; \Omega) \approx \sum_{e_{ab}} \abs{
u^\lambda_a - u^\lambda_b} \frac{\det M(e_{ab}) \norm{e_{ab}}^2
\, \delta^2 \, \Delta\phi}{2 \left(e_{ab}^T \cdot M(e_{ab}) \cdot
e_{ab}\right)^{\sfrac{3}{2}}}.
- \label{eq:per_approx1}
+ \label{eq:per_approx2}
\end{equation}
The absolute value $\abs{u^\lambda_a - u^\lambda_b}$ is one if one of
$a$ and $b$ lie inside the level set and the other lies outside, and
crosses the perimeter of $\{ u > \lambda\}$ an odd number of times, and
zero otherwise.
-\fixme{We now have the problem that $C$ is a curve, but later we want it
-to be a cut...}
+Thus we have arrived at our final discretization, which takes the form
+\begin{gather}
+ F(u) = \sum_\lambda \sum_x F_x^\lambda(u_x^\lambda) + \beta \sum_\lambda
+ \sum_{(x, y)} F_{x,y}^\lambda(u_x^\lambda, u_y^\lambda) :=
+ F^\lambda(u^\lambda), \\
+ \begin{aligned}
+ F_x^\lambda(u_x^\lambda) &= \big(N_x(\lambda + 1) -
+ N_x(\lambda)\big) \cdot u_x^\lambda, \\
+ F_{x,y}^\lambda(u_x^\lambda, u_y^\lambda)
+ &= \abs{u_x^\lambda - u_y^\lambda} \frac{\det M(e_{xy})
+ \norm{e_{xy}}^2 \delta^2 \Delta \phi}{2 \left( e_{xy}^T \cdot
+ M(e_{xy}) \cdot e_{xy} \right)^{\sfrac{3}{2}}}.
+ \end{aligned}
+\end{gather}
+
+If we minimize $F_\lambda$ to obtain $u^\lambda$ for each level
+separately, it is obvious that we will also minimize the sum over all
+$F_\lambda$. However, it is not guaranteed that the obtained thresholded
+images $u^\lambda$ can be combined to make an output image $u$. They
+were defined as $u^\lambda = \idfun_{u > \lambda}$, so we need them to
+be monotonically decreasing (?) in increasing level values, i.e.\
+\begin{equation}
+ u_x^\lambda \geq u_x^\mu, \quad \forall \lambda \leq \mu, \quad
+ \forall x \in \mathcal{G}.
+\end{equation}
+Later we will present a graph cut algorithm that find the thresholded
+images minimizing each level, \emph{while guaranteeing that they meet
+this requirement.}
+
+\fixme{maybe with a $w_{xy}$ definition}
\fixme{edges here are a bit different from edges later}
\subsubsection{Consistency}
Consistency relates to how well a solution to the continuous problem
-fits in the discretized equation, in other words, how well the
+fits in the discretized equation, in other words, whether the
discretized equation approximates the continuous one.
It is obvious that the discretization of the fidelity term in
-\eqref{eq:fidelity_approx_1} is consistent. We have left out the pixel
-size $\Delta x$ in the sum, and absorbed it into the regularization
-parameter $\beta$, but apart from that, the sum is a midpoint rule
-approximation of the integral.
+\eqref{eq:fidelity_approx_1} is consistent. The sum is a midpoint rule
+approximation of the integral. As the grid is refined and $\delta \to 0$
+the sum will converge to the integral.
For the regularization term we will argue that for a differentiable
curve $C$, the discretization of our domain $\Omega$ and the set of
convenience we will use a neighborhood representation of $\mathcal{L}_D$
similar to the one in Figure \ref{fig:line_neigh}.
-\begin{figure}
- \input{fig/square_cons}
-\end{figure}
-
-Consider a square centered around a grid point with side lengths
-$\sqrt{\delta}$ as shown in Figure \ref{fig:square_cons}. As $\delta$
-goes to zero, the size of this square will go to zero. Inside this
-square we can fit a square of $\lfloor 1 / \sqrt{\delta} \rfloor^2$ grid
-points. This means that the number of grid points along the outer edge
-of this square $\lfloor 1 / \sqrt{\delta} \rfloor$ goes to infinity. For
-each grid point along the outer edge of this square, we include in our
-neighborhood a grid point having the same angle $\phi$ to the $x$-axis.
-This means either including the actual grid point at the outer edge, or
-one having the same angle, just closer to the center. This construction
-can be seen in Figure \ref{fig:square_cons}.
-The maximal $\Delta \phi$ will then be between the
-horizontal or vertical edge and its neighbors, shown in Figure
-\ref{fig:square_cons} as angle $a$. These angles can be calculated to be
+If we consider the edges $e$ of each family separately, the curve length
+approximation in \eqref{eq:cauchy_crofton_approx2} can be written
\begin{equation}
- \sup \Delta \phi = \arctan \frac{\delta}{\sfrac{\lfloor \sqrt{\delta}
- \rfloor}{2}} \leq \arctan \frac{\delta}{\sfrac{\sqrt{\delta}}{2} -
- \delta} = \arctan \frac{1}{\frac{1}{2\sqrt{\delta}} - 1} \to 0.
+ \begin{aligned}
+ \abs{C}_M &=
+ \int_\nu \int_\rho \sum_{x \in \ell_{\nu, \rho} \cap C}
+ \, \frac{\det M(x)}{2\left(\nu^T
+ \cdot M(x) \cdot \nu\right)^{\sfrac{3}{2}}} \, d\rho \, d\nu \\
+ &\approx \sum_\nu \sum_\rho \sum_{e_{\nu, \rho} \cap C}
+ \, \frac{\det M(e_{\nu, \rho}) \norm{e_{\nu, \rho}}^3}{2\left(
+ e_{\nu, \rho}^T \cdot M(e_{\nu, \rho}) \cdot e_{\nu, \rho}
+ \right)^{\sfrac{3}{2}}} \, \Delta \rho \, \Delta \nu.
+ \end{aligned}
\end{equation}
-\fixme{the flooring here is not right}
-Further we know from Lemma \ref{lem:delta_rho} that for each line family
-\begin{equation}
- \delta^2 = \Delta \rho \norm{e},
-\end{equation}
-and since the edge length $\norm{e}$ is bounded from below by $\delta$,
-we have
-\begin{equation}
- \sup \Delta \rho \leq \frac{\delta^2}{\delta} = \delta \to 0.
-\end{equation}
+As described in the construction of this formula, there are four main
+approximations used. Firstly there is the fact that we do not consider
+the actual intersection points, but only whether an edge crosses the
+curve or not. Secondly we have the tensor which is averaged as in
+\eqref{eq:tensor_approx}. And then we have the discretizations of our
+two line parameters $\nu$ and $\rho$.
-Since all edges are inside our square,
-we have $\norm{e} \leq \sqrt{2} \delta \to 0$. Thus for a given
-curve $C$, our simplification of only counting an intersection between
-$e$ and $C$ when $C$ crosses $e$ an odd number of times becomes
-increasingly correct.
+It is intuitive that if $\sup \norm{e} \to 0$, the number of times the
+differentiable curve $C$ can cross a given edge decreases. We will not
+prove convergence, but rather assume that the special cases where it
+might not work, are negligible.
-Further it is obvious that our tensor approximation in
-\eqref{eq:tensor_approx} converges to the tensor in the intersection
-point when the edge length $\norm{e_{ab}}$ goes to zero.
+Further, if $\sup \norm{e} \to 0$ it is obvious that the tensor average
+in \eqref{eq:tensor_approx} converges to the tensor in the intersection
+point.
-To see that this is enough to give a consistent discretization of our
-integral, we write the approximation as follows
-\begin{equation}
- \begin{aligned}
- \abs{C}_M &=
- \int_\nu \int_\rho \sum_{x \in \ell_{\nu, \rho} \cap C}
- \, \frac{\det M(x)}{2\left(\nu^T
- \cdot M(x) \cdot \nu\right)^{\sfrac{3}{2}}} \, d\rho \, d\nu \\
- &\approx \sum_\nu \sum_\rho \sum_{e_{\nu, \rho} \cap C}
- \, \frac{\det M(e_{\nu, \rho}) \norm{e_{\nu, \rho}}^3}{2\left(
- e_{\nu, \rho}^T \cdot M(e_{\nu, \rho}) \cdot e_{\nu, \rho}
- \right)^{\sfrac{3}{2}}} \, \Delta \rho \, \Delta \nu.
-\end{aligned}
-\end{equation}
-As $\Delta \rho$ is constant for all lines in one family, we can regard
-the discretization in the $\rho$ dimension as a midpoint rule
-approximation, as shown in Figure \ref{fig:line_midpoint}.
+For each $\phi$ parameter, our discretization in the $\rho$ dimension
+can be regarded as a midpoint rule as shown in Figure
+\ref{fig:line_midpoint}. Thus if $\sup \Delta \rho \to 0$, this part of
+the discretization is fine.
\begin{figure}
\input{fig/line_midpoint}
\end{figure}
-For the $\phi$ dimension (or $\nu$ equivalently), we chose to show that
-the difference between the angle parameter of one line family and the
-next $\Delta \phi$ goes to zero. As shown in Figure \ref{fig:circ_rule},
-the discretization of $\phi$ can also be regarded as a so-called
-\emph{rectangle method} approximation of the integral, although not the
-midpoint rule. The summand is evaluated in the end point of the angle
-interval $[\phi_k, \phi_{k+1}]$, where $\Delta \phi_k = \phi_{k+1} -
-\phi_k$.
+The discretization in the $\phi$ dimension can also be regarded as a
+version of the \emph{rectangle method}, although not the midpoint rule.
+As shown in Figure \ref{fig:circ_rule}, the summand is evaluated on the
+endpoint of the partition intervals $[\phi_k, \phi_{k+1}]$ and the
+difference is taken to be $\Delta \phi = \phi_{k+1} - \phi_k$. Thus if
+$\sup \Delta \phi \to 0$, this discretization is also consistent.
\begin{figure}
\input{fig/circ_rule}
\end{figure}
+To show that all these properties can be fulfilled, we look at a
+particular neighborhood stencil construction.
+Consider a square centered around a grid point with side lengths
+$\sqrt{\delta}$ as shown in Figure \ref{fig:square_cons}. As $\delta$
+goes to zero, the size of this square will go to zero. Inside this
+square we can fit a square of $n^2 = \lfloor 1 / \sqrt{\delta} \rfloor^2$ grid
+points. This means that the number of grid points along the outer edge
+of this square $n$ goes to infinity.
+
+\begin{figure}
+ \input{fig/square_cons}
+\end{figure}
+
+For each grid point along the outer edge of this square, we include in
+our neighborhood a grid point having the same angle $\phi$ to the
+$x$-axis. This means either including the actual grid point at the
+outer edge, or one having the same angle, just closer to the center.
+This construction can be seen in Figure \ref{fig:square_cons} for $n =
+5$. The maximal $\Delta \phi$ will then be between the horizontal or
+vertical edge and its neighbors, shown in Figure \ref{fig:square_cons}
+as angle $a$. These angles can be calculated to be
+\begin{equation}
+ \sup \Delta \phi = \arctan \frac{1/n}{n/2} = \arctan
+ \frac{2}{n^2} \to 0.
+\end{equation}
+
+Further we see that the edge length will be bounded by half of the
+diagonal of the square such that
+\begin{equation}
+ \norm{e} \leq \sqrt{\delta / 2} \to 0.
+\end{equation}
+And finally we know from Lemma \ref{lem:delta_rho} that for each line family
+$\delta^2 = \Delta \rho \norm{e}$ and the fact that $\norm{e} \geq
+\delta$. Thus for the inter-line distance $\Delta \rho$ we have
+\begin{equation}
+ \sup \Delta \rho = \sup \frac{\delta^2}{\norm{e}} \leq
+ \frac{\delta^2}{\delta} = \delta \to 0.
+\end{equation}
+
And thus the approximation has been shown to be equivalent to
-well-known, and consistent integral approximations, so the perimeter
-approximation in \eqref{eq:per_approx1} is consistent with the
-continuous formulation in Theorem \ref{thm:riemannian_cauchy_crofton}.
+well-known, and consistent integral approximations, where the
+summand converges to the integrand, and the differences $\Delta \phi$
+and $\Delta \rho$ go to zero. Thus the perimeter approximation in
+\eqref{eq:per_approx1} is consistent with the continuous formulation in
+Theorem \ref{thm:riemannian_cauchy_crofton}.
Note that as we will work with digital images with fixed resolutions, we
do not really have the chance to refine our discretization. We do
neighborhood stencil, to make sure that we get a reasonable
approximation of the perimeter lengths.
-\fixme{we did not really state that we have arrived at our final
-discretization, and that the rest will be about the solution}
-
\subsubsection{Stability and convergence}
-For a discretization of a problem to be stable, the solution needs to
-depend continuously on the input data. So in our case the output image
-$u$ should depend continuously on the given input image $f$. We will not
-prove stability here, but intuitively there is no reason to believe the
-method is unstable though, as small changes in $f$ only gives small
-changes to the functional $F$. \fixme{well this is not nearly enough I
-guess} \fixme{ref?}
-
-Convergence is in some nice cases, but not nearly all the time,
-equivalent to consistency and stability. It answers the question of
-whether a the solutions of the discretized problem converges to the
-solution of the continuous problem as the discretization is refined.
-This will also not be considered here, but we refer to \fixme{ref}. And
-note that even though convergence is important, in practice, we have no
-possibility of refining the discretization, as the grid structure is
-defined by the digital image which is already composed of discrete
-pixels.
-
-\subsection{Total energy}
-
-By combining the discretized fidelity and regularization terms and
-ignoring the constant term $N_x(0)$ we obtain an energy function which
-is decomposed into a sum over all the levels
-\begin{equation}
- E_v(u) =
- \sum_{\lambda=0}^{L-2} \sum_x E^x_\lambda(u^\lambda_x)
- + \beta \sum_{\lambda = 0}^{L-2} \sum_{(x, y)} E^{x,y}(u^\lambda_x,
- u^\lambda_y)
- =: \sum_{\lambda=0}^{L-2} F_\lambda(u^\lambda)
- \label{eq:total_energy}
-\end{equation}
-\fixme{oops label}
-where
-\begin{align}
- E^x_\lambda(u^\lambda_x) &=
- \big(
- N_x(\lambda + 1) -
- N_x(\lambda)
- \big)
- \, u^\lambda_x
- \label{eq:fidelity_energy} \\
- E^{x,y}(u^\lambda_x, u^\lambda_y) &=
- w_{xy} \abs{u_x^\lambda - u_y^\lambda}.
- \label{eq:neigh_energy}
-\end{align}
-If we minimize $F_\lambda$ to obtain $u^\lambda$ for each level
-separately, it is obvious that we will also minimize the sum over all
-$F_\lambda$. However, it is not guaranteed that the obtained thresholded
-images $u^\lambda$ can be combined to make an output image $u$. They
-were defined as $u^\lambda = \idfun_{u > \lambda}$, so we need them to
-be monotonically decreasing (?) in increasing level values, i.e.\
-\begin{equation}
- u_x^\lambda \geq u_x^\mu, \quad \forall \lambda \leq \mu, \quad
- \forall x \in \mathcal{G}.
-\end{equation}
-In the following we will see two graph cut algorithms that find
-thresholded images minimizing each level, \emph{while guaranteeing that
-they meet this requirement.}
+\fixme{Boop.}
\section{Graph cut formulation}
projects / master / commitdiff
