Replicating a plot using tikz

Possibly the hardest part is in funding a suitable function to plot, but a little thought shows that something like x^2/2-3x+5 works. As you want to reuse this function several times I think that the best approach is to declare the function explicitly using:

begin{tikzpicture}[declare function={f(x)=0.5*x*x-3*x+5;}]
    ...
end{tikzpicture}

after which you can draw points using (a,{f(a)}) for different values of a. The braces around the f(a) are necessary so that tikz does not get confused about the coordinates of the point being plotted. Here a can be anything from an explicit number to a variable in a foreach loop or a plot command.

Once this is done some fairly straightforward tikz commands produce:

Note that, unlike in the image in the OP, I have put the points on top of the function as I think this looks better. To change this all you have to do is move the plot command after the foreach loop. I have left the addition of the words "Minimum" and "Learning step" as an exercise:)

Here is the code:

documentclass{article}
usepackage{tikz}
usetikzlibrary{arrows.meta}
begin{document}

tikzset{
  point/.style = {% define a style for the function points
    circle,
    fill=#1,
    draw=black,
    inner sep=2pt,
  },
  point/.default = {violet!60}
}

begin{tikzpicture}[declare function={f(x)=0.5*x*x-3*x+5;}]
    draw[thick,-{LaTeX}](0,0)--(6,0) node[right]{$theta$};
    draw[thick,-{LaTeX}](0,0)--(0,4) node[above]{Cost};
    draw[thick, dashed](0.6, {f(0.6)}) -- (0.6,0)
          node[below, text width=4em, align=center]{Random initial value};
    draw[thick, dashed](3, {f(3)}) -- (3,0) node[below]{$hattheta$};
    draw[domain=0.5:5.5, smooth, thick, gray] plot (x, f(x);
    foreach a [count=c, remember=c as C] in {0.6, 1.0, ..., 2.6} {
      node[point] (c) at (a, {f(a)}){};
      ifnumc>1 % after the first coordinate draw an arrow
         draw[->, bend left](C) to (c);
      fi
    }
    node[point=yellow] (Y) at (3, {f(3)}){}; % add the yellow point
    draw[->, bend left](6) to (Y);
 end{tikzpicture}

end{document}

More points can be added simply by changing the list of points in the foreach loop (and then updating the (6) to the number of points plotted in the last draw command).