\label{fig:fn_resid}\r
\end{figure}\r
\r
-\begin{figure}[b]\r
- \centering\r
- \includegraphics[width=0.9\textwidth]{surface}\r
- \caption{\sf Surface plot of function with points showing iteration of steepest\r
- descent method, both with optimal and constant $\alpha$}\r
- \label{fig:surface}\r
-\end{figure}\r
-\r
The steepest descent method works by always taking a step in the direction\r
opposite the gradient in the point considered (i.e. the direction of the\r
steepest descent). The step size can be set to a constant or chosen in a more\r
\mathbf{\alpha}}g\left(\mathbf{x}+\alpha\mathbf{u}\right) &=& 0, \text{where }\r
\mathbf{u} = -\nabla g(\mathbf{x} ) \nonumber \\\r
&\Downarrow& \nonumber \\\r
- A\alpha^3 + B\alpha^2 + C\alpha + D &=& 0,\r
+ A\alpha^3 + B\alpha^2 + C\alpha + D &=& 0.\r
\end{eqnarray}\r
Where $A$, $B$, $C$, and $D$ are given by\r
\begin{eqnarray}\r
A &=& \frac{1}{3}\mathbf{u}^TC(\mathbf{u})\mathbf{u} \nonumber \\\r
B &=& \mathbf{x}^TC(\mathbf{u}) \mathbf{u} \nonumber \\\r
C &=& \mathbf{u}^TH\mathbf{u}+\mathbf{u}^TC(\mathbf{x})\mathbf{u} \nonumber \\\r
- D &=& \frac{1}{3}\mathbf{u}^TC(\mathbf{x})\mathbf{x}+\mathbf{u}^TH\mathbf{x}-\mathbf{b}^T\mathbf{u}\r
+ D &=& \frac{1}{3}\mathbf{u}^TC(\mathbf{x})\mathbf{x}+\mathbf{u}^TH\mathbf{x}-\mathbf{b}^T\mathbf{u}.\r
\end{eqnarray}\r
Differentiating the polynomial results in:\r
\begin{eqnarray}\r
computing the optimal step size requires solving a cubic equation and is\r
computationally expensive.\r
\r
+\begin{figure}[hb]\r
+ \centering\r
+ \includegraphics[width=0.9\textwidth]{surface}\r
+ \caption{\sf Surface plot of function with points showing iteration of steepest\r
+ descent method, both with optimal and constant $\alpha$}\r
+ \label{fig:surface}\r
+\end{figure}\r
+\r
\end{document}\r