trying to argue how the choices we make can have a positive impact on
the image restoration results. The anisotropic total variation is the
main advance from my project work \cite{project}, and we would like to
-see how it affects the performance of the restoration algorithm.
+see how the introduction of the anisotropy affects the performance of
+the restoration algorithm.
\section{Tensor parameters}
Normal tensor.
}
\end{subfigure}
- \caption{%
+ \caption[Restoration using a uniform tensor]{%
A noisy circle first restored using a uniform tensor, resulting
mostly in smoothing in the $x$-direction. Then restored using
the tensor described in Section~\ref{sec:anisotropy_tensor}.
predescribed tensor affects the regularization. Imagine a tensor which
is a diagonal matrix $A = \operatorname{diag}(1, \epsilon)$, where
$\epsilon \ll 1$ is small. This would result in us down-weighting the
-size of $\nabla f_\sigma$ in the $y$-direction, and thus regularization
+size of $\nabla u$ in the $y$-direction, and thus regularization
mostly in the $x$-direction.
The results of this experiment can be seen in
}
\label{fig:lena_process_restored}
\end{subfigure}
- \caption{%
+ \caption[The different stages of the restoration algorithm
+ visualized]{%
The different stages of the restoration algorithm, showing the
original image, the smoothed image, the edge detector, two
visualizations of the anisotropy tensor, and finally the
The blurring is done so that the edge detector $\nabla f_\sigma$, which
is shown squared and normalized in Figure~\ref{fig:lena_process_edge}, is not too
sensitive to noise in the image. The structure tensor is then
-constructed using $\nabla f_\sigma$, and further transformed into the
-anisotropy tensor, which is visualized in
+constructed using $\nabla f_\sigma$ and smoothed according to the
+integration scale $\rho$.
+The resulting anisotropy tensor is visualized in
Figure~\ref{fig:lena_process_tensor}. In a selection of points, the
tensor has been drawn as its two eigenvectors, with the corresponding
eigenvalue as its length, and we clearly see that the eigenvalues are
it possible to see it in every point, is using the color wheel in
Figure~\ref{fig:color_wheel}. The color is decided by the tensor angle,
while the brightness, or the radius in the wheel, is set to $1 /
-\lambda_2$, the inverse of the smallest eigenvalue. Thus the stronger
-the anisotropy, the brighter the color, while for smooth areas we expect
-black.
+\lambda_2$, the inverse of the smallest eigenvalue of the metric tensor
+$M(x)$. Thus the stronger the anisotropy, the brighter the color, while
+we expect smooth areas in the original image to be black.
\begin{figure}
\centering
\includegraphics[width=0.3\textwidth]{fig/wheel.png}
- \caption{%
- Color wheel.
+ \caption[Color wheel]{%
+ Color wheel used for tensor visualization.
}
\label{fig:color_wheel}
\end{figure}
Finally, Figure~\ref{fig:lena_process_restored} shows the restored
-image, and we see that it has been heavily regularized.
+image, and we see that it has been heavily regularized. How the
+anisotropy affected the regularization is not obvious however.
\begin{figure}
\centering
}
\label{fig:scale_comp_int}
\end{subfigure}
- \caption{%
+ \caption[Effects of the $\sigma$ and $\rho$ parameters on
+ restoration]{%
An example constructed to show the effects of the parameters
$\sigma$ and $\rho$ in the anisotropy tensor. The noise scale
$\sigma$ controls what is considered to be noise, while the
as shown in Figure~\ref{fig:scale_comp}.
The anisotropy introduced should in theory help reduce contrast
loss in this situation, by reducing the regularization done in the
-$x$-direction, but there is the question of how the different scales
-affect the regularization.
+$x$-direction across the edges. There is however the question of how the
+different scales $\sigma$ and $\rho$ affect the regularization.
In Figure~\ref{fig:scale_comp_high} the noise scale is increased such
that the edge detector, and thus the anisotropy tensor, considers the
\centering
\begin{subfigure}[t]{0.30\textwidth}
\centering
- \includegraphics[width=\textwidth]{fig/factory/finger/n_q300.png}
+ \includegraphics[width=\textwidth]{fig/factory/finger/n_q200.png}
\caption{%
Noisy fingerprint.
}
+ \label{fig:noisy_fingerprint}
\end{subfigure}
~
\begin{subfigure}[t]{0.30\textwidth}
\centering
- \includegraphics[width=\textwidth]{fig/factory/finger/r_p2_n32_b9000_g80_r10_s3_color.png}
+ \includegraphics[width=\textwidth]{fig/factory/finger/r_p2_n32_b15000_g150_r10_s3_color.png}
\caption{%
$\rho = 10$
}
~
\begin{subfigure}[t]{0.30\textwidth}
\centering
- \includegraphics[width=\textwidth]{fig/factory/finger/r_p2_n32_b9000_g80_r20_s3_color.png}
+ \includegraphics[width=\textwidth]{fig/factory/finger/r_p2_n32_b15000_g150_r20_s3_color.png}
\caption{%
$\rho = 20$
}
\end{subfigure}
- \caption{%
+ \caption[Anisotropy tensor visualized for a fingerprint test image]{%
A noisy fingerprint with the anisotropy tensor visualized for
different integration scales $\rho$.
}
~
\begin{subfigure}[t]{0.30\textwidth}
\centering
- \includegraphics[width=\textwidth]{fig/factory/circle_deform/r_p2_n8_b100000_g10000000000_r1_s1.png}
+ \includegraphics[width=\textwidth]{fig/factory/circle_deform/r_p2_n8_b200000_g10000000000_r1_s1.png}
\caption{%
$\abs{\mathcal{N}} = 8$.
}
~
\begin{subfigure}[t]{0.30\textwidth}
\centering
- \includegraphics[width=\textwidth]{fig/factory/circle_deform/r_p2_n72_b100000_g10000000000_r1_s1.png}
+ \includegraphics[width=\textwidth]{fig/factory/circle_deform/r_p2_n72_b200000_g10000000000_r1_s1.png}
\caption{%
$\abs{\mathcal{N}} = 72$.
}
~
\begin{subfigure}[t]{0.30\textwidth}
\centering
- \includegraphics[width=\textwidth]{fig/factory/octagon/r_p2_n8_b100000_g10000000000_r1_s1.png}
+ \includegraphics[width=\textwidth]{fig/factory/octagon/r_p2_n8_b200000_g10000000000_r1_s1.png}
\caption{%
$\abs{\mathcal{N}} = 8$.
}
~
\begin{subfigure}[t]{0.30\textwidth}
\centering
- \includegraphics[width=\textwidth]{fig/factory/octagon/r_p2_n72_b100000_g10000000000_r1_s1.png}
+ \includegraphics[width=\textwidth]{fig/factory/octagon/r_p2_n72_b200000_g10000000000_r1_s1.png}
\caption{%
$\abs{\mathcal{N}} = 72$.
}
~
\begin{subfigure}[t]{0.30\textwidth}
\centering
- \includegraphics[width=\textwidth]{fig/factory/rombus/r_p2_n4_b100000_g10000000000_r1_s1.png}
+ \includegraphics[width=\textwidth]{fig/factory/rombus/r_p2_n4_b200000_g10000000000_r1_s1.png}
\caption{%
$\abs{\mathcal{N}} = 4$.
}
~
\begin{subfigure}[t]{0.30\textwidth}
\centering
- \includegraphics[width=\textwidth]{fig/factory/rombus/r_p2_n72_b100000_g10000000000_r1_s1.png}
+ \includegraphics[width=\textwidth]{fig/factory/rombus/r_p2_n72_b200000_g10000000000_r1_s1.png}
\caption{%
$\abs{\mathcal{N}} = 72$.
}
\label{fig:n_square_72}
\end{subfigure}
- \caption{%
+ \caption[Shapes restored to show artifacts introduced by
+ neighborhood choice]{%
Different test images restored without anisotropy, different
neighborhoods and a large restoration parameter $\beta$. We see
how different neighborhood stencils introduce different
Figure~\ref{fig:neigh_shapes} shows three images of different shapes
that are heavily regularized using different neighborhood stencils. In
these results we see some of the artifacts that may occur because of
-this discretization method. It is clear that the size-8 stencil prefers
-horizontal, vertical and 45\textdegree{} lines, such that the circle in
+this particular discretization method. It is clear that the stencil of
+size 8 prefers
+horizontal, vertical and 45\textdegree{} perimeters, such that the circle in
Figure~\ref{fig:n_circle} is shaped like an octagon when regularized in
-Figure~\ref{fig:n_circle_8}. The neighborhood of size 72 manages to
-keep the circular shape however.
+Figure~\ref{fig:n_circle_8}. However the stencil of size 72 manages to
+keep the circular shape.
-Next, in Figure~\ref{fig:n_octagon}, we see that the size-8 stencil
+Next, in Figure~\ref{fig:n_octagon}, we see that the stencil of size 8
actually favors octagon-like shapes, as the restored shape is the same
octagon, just with some contrast loss.
-The tilted square in Figure~\ref{fig:n_square}, hints that a size-4
-neighborhood favors horizontal and vertical edges only. While the larger
-size-72 neighborhood retains the square shape but smoothes the corners
-some.
+The tilted square in Figure~\ref{fig:n_square}, hints that a stencil of
+size 4 neighborhood favors horizontal and vertical edges only. While the
+larger stencil of size 72 retains the square shape but rounds the
+corners some.
\begin{figure}
\centering
\begin{subfigure}[t]{0.46\textwidth}
\centering
- \includegraphics[trim=240 170 110 180,clip=true,width=\textwidth]{fig/factory/lena_neigh/r_p2_n16_b2000_g80_r10_s5.png}
+ \includegraphics[trim=240 170 110 180,clip=true,width=\textwidth]{fig/factory/lena_neigh/r_p2_n16_b4000_g80_r10_s5.png}
\caption{%
$\abs{\mathcal{N}} = 16$
}
~
\begin{subfigure}[t]{0.46\textwidth}
\centering
- \includegraphics[trim=240 170 110 180,clip=true,width=\textwidth]{fig/factory/lena_neigh/r_p2_n72_b2000_g80_r10_s5.png}
+ \includegraphics[trim=240 170 110 180,clip=true,width=\textwidth]{fig/factory/lena_neigh/r_p2_n72_b4000_g80_r10_s5.png}
\caption{%
$\abs{\mathcal{N}} = 72$
}
\end{subfigure}
- \caption{%
+ \caption[Noisy Lena restored using different neighborhood stencils]{%
Noisy Lena restored using different neighborhood stencils. Note
how the large neighborhood introduces some pixel size artifacts.
}
\end{figure}
\begin{table}
- \caption{%
+ \caption[Circle circumferences estimated by the Cauchy--Crofton
+ formula]{%
The circumference of circles of different radii $r$ measured by the
- discretized Cauchy--Crofton formula in \eqref{ref}, using different
+ discretized Cauchy--Crofton formula in
+ \eqref{eq:final_discretization}, using different
neighborhood stencils.
}
\centering
- u^\lambda_b}$, an approximation that becomes worse for long edges. Thus a larger
neighborhood is not always better, even if it will reduce the artifacts
discussed above. In Figure~\ref{fig:lena_neigh}, a noisy image of Lena
-has been restored using two differently sized neighborhood stencils, and
+has been restored using two neighborhood stencils, and
there are obvious differences. For the stencil of size 72, the restored
-image still contains some pixel-sized noise, seemingly very
-different from their neighboring pixels. An explanation can be found in
-Figure~\ref{fig:pixel_perimeter}. We see that the length of a
-one-pixel curve is underestimated by the Cauchy--Crofton formula because
-many edges cross the curve cross while $\abs{u^\lambda_a - u^\lambda_b}
-= 0$, and thus these edges are ignored completely in our perimeter
-approximation. An additional demonstration that this problem mostly relates
+image still contains some pixel-sized noise.
+An explanation can be found in
+Figure~\ref{fig:pixel_perimeter}. We see why the length of a
+one-pixel curve is underestimated by the Cauchy--Crofton formula when
+some of the edges are long. Because
+many edges $e_{ab}$ cross the curve cross while $\abs{u^\lambda_a - u^\lambda_b}
+= 0$, thus these edges are ignored completely in our perimeter
+approximation.
+
+An additional demonstration that this problem mostly relates
to small sized noise is shown in Table~\ref{tab:circumference}. The
table shows how our discrete Cauchy--Crofton formula approximates the
circumference of circles of different radii. Note that the circumference
-approximated is that of an actual continuous circle $r = x^2 + y^2$ and
-not a discrete representation.
+approximated is that of an actual continuous circle $u : \mathbb{R}^2
+\to \{0, 1\}$ and not a discrete representation.
\section{Restoration}
\centering
\begin{subfigure}[t]{0.30\textwidth}
\centering
- \includegraphics[width=\textwidth]{fig/factory/finger/n_q300.png}
+ \includegraphics[width=\textwidth]{fig/factory/finger/r_p2_n32_b15000_g1000000000000_r10_s3.png}
\caption{%
- Noisy fingerprint.
+ Regular TV,\\ $\beta=15000$,\\ $\norm{u - f}_{L^1} =
+ 2886817$.
}
+ \label{fig:finger_contrast_tv}
\end{subfigure}
~
\begin{subfigure}[t]{0.30\textwidth}
\centering
- \includegraphics[width=\textwidth]{fig/factory/finger/r_p2_n32_b9000_g1000000000000_r10_s3.png}
+ \includegraphics[width=\textwidth]{fig/factory/finger/r_p2_n32_b15000_g150_r10_s3.png}
\caption{%
- Restored without anisotropy.
+ Anisotropic TV ($\gamma = 150$), $\beta=15000$,
+ $\norm{u - f}_{L^1} = 2558191$.
}
- \label{fig:finger_contrast_tv}
+ \label{fig:finger_contrast_atv1}
\end{subfigure}
~
\begin{subfigure}[t]{0.30\textwidth}
\centering
- \includegraphics[width=\textwidth]{fig/factory/finger/r_p2_n32_b9000_g150_r10_s3.png}
+ \includegraphics[width=\textwidth]{fig/factory/finger/r_p2_n32_b21450_g150_r10_s3.png}
\caption{%
- Restored with much anisotropy.
+ Anisotropic TV ($\gamma = 150$), $\beta=21450$,
+ $\norm{u - f}_{L^1} = 2886529$.
}
- \label{fig:finger_contrast_atv}
+ \label{fig:finger_contrast_atv2}
\end{subfigure}
- \caption{%
+ \caption[Comparison contrast loss in regular an anisotropic restoration]{%
+ \fixme{nope}
A noisy fingerprint restored using both isotropic and
anisotropic total variation. All parameters are kept constant
except for $\omega$, the anisotropy parameter.
We have claimed that the anisotropic total variation will reduce some of
the contrast loss one can encounter with regular total variation
-regularization. In Figure~\ref{fig:finger_contrast} a fingerprint with
-additional Gaussian noise has been restored with two different
-anisotropy parameters. There is an obvious contrast difference in the
-contrast of the two restored images, which can also be seen in
-Figure~\ref{fig:contrast_plot} where a one-dimensional slice has been
-taken through Figure~\ref{fig:finger_contrast_tv}
-and~\ref{fig:finger_contrast_atv}. It is however not obvious what this
+regularization. In Figure~\ref{fig:finger_contrast} the noisy
+fingerprint from Figure~\ref{fig:noisy_fingerprint}
+has been restored three times with different parameters. The first using
+regular total variation and $\beta = 15000$, then the second with $\beta
+= 15000$ and anisotropy
+$\omega = 150$. This gives an obvious contrast enhancement,
+but it is however not obvious what this
tells us about the quality of the anisotropic algorithm. As previously
mentioned, increasing the anisotropy (decreasing $\omega$) means
-decreasing the amount of regularization done. And less contrast loss is
+decreasing the total amount of regularization done, and less contrast loss is
an expected outcome of decreasing the regularization, disregarding the
anisotropy.
+Thus in the last
+image we have also used $\omega = 150$ but the restoration
+amount $\beta$ is increased such that the amount of noise removed
+$\norm{u - f}_{L^1}$ is
+approximately equal to the noise removed when using regular total
+variation in Figure~\ref{fig:finger_contrast_tv}. By visual inspection,
+the last image seems to have somewhat higher contrast. This can be
+confirmed by calculating the standard deviation of the images, one of
+many possible contrast measures, which gives 22.9, 37.9 and 26.6
+respectively.
+
+To further compare the three restored images, a single row has been
+extracted from the three images and is shown agains eachother in
+Figure~\ref{fig:contrast_plot}. Note that although the results in
+Figure~\ref{fig:finger_contrast_atv2} look promising compared to
+Figure~\ref{fig:finger_contrast_tv}, some details in the singularity in
+the upper left is actually lost. The metric tensor is approximately the
+identity matrix there, and thus an increased restoration parameter
+$\beta$ leads to increased smoothing.
+
\begin{figure}
\centering
\input{fig/factory/contrast}
- \caption{%
+ \caption[1D slice through restored fingerprints showing contrast
+ differences]{%
A one-dimensional slice through the restored fingerprint images
of Figure~\ref{fig:finger_contrast}, showing a difference in the
contrast.
+ \frac{(s_1 - s_2)^2}{\omega^2}\right)^{-1} \geq \left(1 +
\frac{s_1^2}{\omega^2}\right)^{-1} \geq k > 0.
\end{equation}
-Hence, our metric tensor $M(x) = P A(x) P^T$ and curve length
-calculation in
-\eqref{eq:per_to_length2} fulfill all the assumptions of the anisotropic
-Cauchy--Crofton formula in Theorem~\ref{thm:riemannian_cauchy_crofton}.
-\fixme{did we argue for
-continuous? and pos def?} Thus we can apply the formula to calculate the
+Hence our metric tensor $M(x) = P A(x) P^T$ is continuous and positive
+definite and thus the curve length calculation in
+\eqref{eq:per_to_length2} fulfills all the assumptions of the
+anisotropic Cauchy--Crofton formula in
+Theorem~\ref{thm:riemannian_cauchy_crofton}.
+Thus we can apply the formula to calculate the
perimeter in \eqref{eq:per_to_length2} as
\begin{equation}
\PerA(U; \Omega) = \int_\mathcal{L} \sum_{x \in
where
\begin{equation}
\TVA(u) = \int_{-\infty}^\infty \int_{\mathcal{L}} \sum_{x \in
- \ell_{\nu, \rho} \cap C_s} \frac{\det M(x)}
+ \ell_{\nu, \rho} \cap \gamma_s} \frac{\det M(x)}
{2 \left( \nu^T \cdot M(x) \cdot \nu \right)^{\sfrac{3}{2}}}
- \, d\mathcal{L}(\ell_{\nu, \rho}) \, ds.
+ \, d\mathcal{L}(\ell_{\nu, \rho}) \, ds,
\end{equation}
-\fixme{define $C_s$. $C_s$ vs $\gamma$.}
+and $\gamma_s = \partial \{u > s\} \cap \Omega$.
Within the restrictions that these theorems put on the tensor $M(x)$, we
have chosen a construction where one eigenvalue is always 1, while the
other varies from 1 in smooth areas towards 0 around edges, with the
\norm{e_{xy}}^2 \delta^2 \Delta \phi}{2 \left( e_{xy}^T \cdot
M(e_{xy}) \cdot e_{xy} \right)^{\sfrac{3}{2}}}.
\end{aligned}
+ \label{eq:final_discretization}
\end{gather}
Recall that $N_x(\lambda) = \abs{ \lambda - f_x }^2$.
neighborhood stencil, to make sure that we get a reasonable
approximation of the perimeter lengths.
-\subsubsection{Stability and convergence}
-
-\fixme{Boop.}
-
\section{Graph cut formulation}
-Maybe this is more tightly connected with the previous section, but the
-main point is that we describe how the edge weights are computed.
+The discretization we arrived at in \eqref{eq:final_discretization} can
+be minimized using graph cuts. For each level $\lambda$, a minimum graph
+cut is found to produce the corresponding level set $\{ u > \lambda \}$. These
+are then combined to form the final restored image $u$.
+
+In this section we will look at how these graphs are constructed such
+that their minimum cuts correspond to the minimizers of the functional
+$F^\lambda$. The description is taken from my project work
+\cite{project}, and is included here for completeness.
\subsection{Graphs}
-\fixme{Maybe just call them graphs this time?}
+\fixme{Maybe just call them graphs this time? from $E$ to $F$!}
Using the notation of \cite{cormen2009introduction} we will denote a
directed graph as $G = (V, E)$ where $V$ is a finite set of vertices, and
minimizing the energy function for one level value $\lambda$. We will in
this and the next section see how such a minimum cut can be found by
sending flow through the graph and trying to identify the
-``bottleneck''.
+``bottleneck''. This chapter, except for the description of the
+Boykov--Kolmogorov algorithm is taken from my project work
+\cite{project} and is included here for completeness.
\section{Flow graphs}
+
We have already introduced capacities, and briefly mentioned the notion
of flow as something limited by the capacity. In other words, flow is
something we can send through our graph, but the capacity limits how
projects / master / commitdiff