Better derivation for the gravitational potential energy

You're just having an issue with improper integrals. All you need to do is use limits, since you can prove that $lim_{rtoinfty}1/r$ goes to $0$.

begin{align} W &= int F dx &= lim_{btoinfty}int_{b}^{r} F , dr' &=lim_{btoinfty} int_{b}^{r} left(frac{G M m}{r'^2}right),dr' &= G M mlim_{btoinfty}int_{b}^{r} frac{dr'}{r'^2} &= G M mlim_{btoinfty}int_{b}^{r} r'^{-2} , dr' &= G M mlim_{btoinfty}left[frac{r'^{-1}}{-1}right]_{b}^r &= - G M mlim_{btoinfty}left[frac{1}{r} - frac{1}{b}right] &= -frac{G M m}{r}+0 U &= -frac{G M m}{r} end{align}

Alternatively, if you just hate using $infty$ altogether, we can be "more physical" by replacing $infty$ with a distance that is "much larger" than $r$. Indeed, you can do the integration to get to

begin{align} U &= -frac{G M m}{r}+frac{G M m}{b} &=-frac{G M m(r-b)}{rb} end{align}

and then assume that $bgg r$ so that

$$frac{r-b}{rb}to-frac{1}{r}$$