The gradient descent I learnt uses $x^{k+1} = x^k + t\triangledown f(x)$ and we learnt to set $t$ heuristically. Am I right to say that exact line search simply computes the optimal value of $t$ that minimizes the $f(x) ?$
Wouldn't I be able to look for the global minima in 1 iteration in that case ? I can't see the negatives of this algorithm eg being stuck in a local minima. Can someone give an example ?
$\endgroup$ 41 Answer
$\begingroup$Exact line search computes the global solution of the one-dimensional problem $$ \min_{t\in \mathbb R} f(x_k - t\nabla f(x_k)). $$ This might be easy if $f$ has nice structure (quadratic).
If the global minima of $f$ are not on the line $x_k - t\nabla f(x_k)$, then you will not find the global minimum in the current step.
$\endgroup$ 2
