Integral: $int dfrac{dx}{(x^2-4x+13)^2}$?

$begin{aligned} I&=int frac{d x}{left[(x-2)^{2}+9right]^{2}} \ &=int frac{d y}{left(y^{2}+9right)^{2}}, text { where } y=x-2 \ &=-frac{1}{2} int frac{1}{y} dleft(frac{1}{y^{2}+9}right) quad text{(By IBP)}\ &--frac{1}{2 yleft(y^{2}+9right)}-frac{1}{2} intleft(frac{1}{y^{2}} cdot frac{1}{y^{2}+9}right) d y \ &=-frac{1}{2 yleft(y^{2}+9right)}-frac{1}{18} intleft(frac{1}{y^{2}}-frac{1}{y^{2}+9}right) d y \ &=-frac{1}{2 yleft(y^{2}+9right)}+frac{1}{18 y}+frac{1}{54} tan ^{-1}left(frac{y}{3}right)+C \ &=frac{y}{18left(y^{2}+9right)}+frac{1}{54} tan ^{-1}left(frac{y}{3}right)+C \ &=frac{1}{54}left(frac{3(x-2)}{x^{2}-4 x+13}+tan ^{-1}left(frac{x-2}{3}right) right)+C end{aligned}$

Substitute $t=x-2$

begin{align} int dfrac{dx}{(x^2-4x+13)^2} & =int dfrac{dt}{(t^2+9)^2}= int frac1{18t}dleft( frac{t^2}{t^2+9}right)\ &= frac t{18(t^2+9)}+frac1{18}int frac{dt}{t^2+9}\ &= frac t{18(t^2+9)}+frac1{54}tan^{-1}frac t3+C end{align}

By setting $$ frac{1}{left(x^2-4 x+13right)^2}=frac{A (2 x-4)+B}{x^2-4 x+13}+frac{d}{dx}left(frac{C x+D}{x^2-4 x+13}right) $$ you get begin{align} A &= 0,\ B &= frac{1}{18},\ C &= frac{1}{18},\ D &= -frac{1}{9} end{align} so that begin{align} intfrac{1}{left(x^2-4 x+13right)^2}dx &= frac{1}{18}intfrac{1}{x^2-4 x+13}dx+frac{x-2}{18(x^2-4 x+13)}=\ &= frac{1}{54} arctanleft(frac{x-2}{3}right)+frac{x-2}{18(x^2-4x+13)}+c end{align}

Let us ise Euler's transformation $$I=int frac{dx}{(x^2-4x+13)^2}=int frac{dx}{(x-a)^2 (x-b)^2},~~a,b=2pm 3i.$$ Let $$t=frac{x-a}{x-b} implies x=frac{bt-a}{t-1} implies dx=frac{a-b}{(t-1)^2}.$$ Then $$I=(b-a)^{-3} int frac{dt}{t^2(t-1)^2}=(a-b)^{-3}int frac{u^2 du}{(u-1)^2}, u=1/t .$$ Next use $u=v+1$, then $$I=(a-b)^3 int [1-2/v+1/v^2] dv= (a-b)^{-3}[v-2 ln v -1/v] $$ $$I=(a-b)^{-3}left(frac{a-b}{x-a}-2ln frac{a-b}{x-a}-frac{x-a}{a-b}right)$$

After square completion and substituting $u=frac{x-2}{3}$, there is a simple standard trick to evaluate the integral without trigonometric substitutions:

$$int dfrac{dx}{(x^2-4x+13)^2} stackrel{u=frac{x-2}{3}}{=}frac 1{27} underbrace{int frac{1}{(u^2+1)^2}du}_{I(u)}$$

Just rewrite the numerator

$$I(u) = intfrac{1+u^2-u^2}{(u^2+1)^2}du = arctan u - frac 12underbrace{int u frac{2u}{(u^2+1)^2}}_{J(u)}$$

So, only one quick partial integration gives

$$J(u) = -frac u{u^2+1}+arctan u$$

Hence,

$$I(u) = arctan u - frac 12left(-frac u{u^2+1}+arctan uright) =frac 12 left(arctan u + frac u{u^2+1}right)$$

Finally, substitute back $u=frac{x-2}{3}$ and you are done:

$$int dfrac{dx}{(x^2-4x+13)^2} = frac 1{27}I(u)= frac 1{54}left(arctan frac{x-2}{3} + frac{frac{x-2}{3}}{left(frac{x-2}{3}right)^2+1}right) (+C)$$ $$= frac 1{54}left(arctan frac{x-2}{3} + frac{3(x-2)}{left(x-2right)^2+9}right)(+C)$$

Here is an alternative method. You can first use substitution to simplify the integral:

$$int dfrac{dx}{(x^2-4x+13)^2}$$ $$ =int dfrac{dx}{((x-2)^2+9)^2} = int dfrac{du}{(u^2+9)^2} tag{$u=x-2, du = dx$}$$ $$=frac{1}{81}int dfrac{du}{(frac{u^2}{9}+1)^2} = frac{1}{27} int dfrac{dv}{(v^2+1)^2} tag{$u=3v, du = 3 dv$}$$

and then use integration by parts as shown in Rene's answer in this thread. This leads to:

$$2int frac{1}{(1+v^2)^2},dv=frac{v}{1+v^2}+int frac{1}{1+v^2},dv$$ $$int frac{1}{(1+v^2)^2},dv=frac{1}{2} left(frac{v}{1+v^2}+arctan(v) right)$$

and then a couple of back-substitutions leads you to the answer of the original problem.

You can solve it by using induction formula: $color{blue}{int frac{dt}{(t^2+a^2)^n}=frac{t}{2(n-1)a^2(t^2+a^2)^{n-1}}+frac{2n-3}{2(n-1)a^2}int frac{dt}{(t^2+a^2)^{n-1}}}$ as follows $$int dfrac{dx}{(x^2-4x+13)^2}$$$$=int dfrac{d(x-2)}{((x-2)^2+3^2)^2}$$ $$=frac{x-2}{2cdot 3^2((x-2)^2+3^2)}+frac{1}{2cdot 3^2}int frac{d(x-2)}{(x-2)^2+3^2}$$ $$=frac{x-2}{18(x^2-4x+13)}+frac{1}{18}left(frac13tan^{-1}left(frac{x-2}{3}right) right)+C$$ $$=bbox[15px,#ffd,border:1px solid green]{frac{x-2}{18(x^2-4x+13)}+frac{1}{54}tan^{-1}left(frac{x-2}{3}right)+C}$$

Since you selected $$x - 2 = 3 tan theta$$ as your substitution, it follows that $$tan theta = frac{x-2}{3},$$ and by considering a right triangle with legs $3$ and $x-2$ with hypotenuse $sqrt{3^2 + (x-2)^2}$ via the Pythagorean theorem, we obtain $$sin theta = frac{x-2}{sqrt{3^2 + (x-2)^2}}, \ cos theta = frac{3}{sqrt{3^2 + (x-2)^2}}.$$ Therefore, $$frac{1}{2} sin 2theta = sin theta cos theta = frac{3(x-2)}{3^2 + (x-2)^2} = frac{3(x-2)}{x^2 - 4x + 13}.$$ We also easily have $$theta = tan^{-1} frac{x-2}{3}.$$ Therefore $$frac{1}{54}left( theta + frac{1}{2} sin 2theta right) = frac{1}{54} left( tan^{-1} frac{x-2}{3} + frac{3(x-2)}{x^2 - 4x + 13} right).$$

substitute $theta=tan^{-1}left(frac{x-2}{3}right)$ & $$sin2theta=frac{2tantheta}{1+tan^2theta}=frac{2left(frac{x-2}{3}right)}{1+left(frac{x-2}{3}right)^2}=frac{6(x-2)}{x^2-4x+13}$$

After substituting $theta$ and $sin2theta$, you will get final answer $$I=frac{1}{54}tan^{-1}left(frac{x-2}{3}right)+frac{x-2}{18(x^2-4x+13)}+C$$