Closed form of $int_0^infty arctan^2 left (frac{2x}{1 + x^2} right ) , dx$

Here is a solution based on Fubini's theorem.

According to an addition formula begin{equation*} arctanleft(dfrac{2x}{1+x^2}right) = arctan((sqrt{2}+1)x)-arctan((sqrt{2}-1)x) . end{equation*} Furthermore begin{equation*} arctan x=mathrm{sign}(x)dfrac{pi}{2}-arctandfrac{1}{x} . end{equation*} Consequently begin{equation*} arctanleft(dfrac{2x}{1+x^2}right) = arctandfrac{sqrt{2}+1}{x}-arctandfrac{sqrt{2}-1}{x}=int_{sqrt{2}-1}^{sqrt{2}+1}dfrac{x}{x^2+s^2}, ds . end{equation*} Via Fubini's theorem we get begin{gather*} int_{0}^{infty}arctan^2left(dfrac{2x}{1+x^2}right), dx = int_{0}^{infty}left(arctandfrac{sqrt{2}+1}{x}-arctandfrac{sqrt{2}-1}{x}right)^2, dx=\[2ex] int_{0}^{infty}left(int_{sqrt{2}-1}^{sqrt{2}+1}dfrac{x}{x^2+s^2}, dsint_{sqrt{2}-1}^{sqrt{2}+1}dfrac{x}{x^2+t^2}, dtright), dx=\[2ex] int_{sqrt{2}-1}^{sqrt{2}+1}left(int_{sqrt{2}-1}^{sqrt{2}+1}left(int_{0}^{infty}dfrac{x^2}{(x^2+s^2)(x^2+t^2)}, dxright), dsright), dt=\[2ex] dfrac{pi}{2}int_{sqrt{2}-1}^{sqrt{2}+1}left(int_{sqrt{2}-1}^{sqrt{2}+1}dfrac{1}{s+t}, dsright), dt=\[2ex] dfrac{pi}{2}int_{sqrt{2}-1}^{sqrt{2}+1}left(ln(t+sqrt{2}+1)-ln(t+sqrt{2}-1)right), dt=\[2ex] 2piln(sqrt{2}+1)-sqrt{2}piln 2. end{gather*}

Remark. Since begin{equation*} arctanleft(dfrac{2xsinhalpha}{1+x^2}right)=arctanleft(dfrac{e^{alpha}}{x}right)-arctanleft(dfrac{e^{-alpha}}{x}right) = int_{e^{-alpha}}^{e^{alpha}}dfrac{x}{x^2+s^2}, ds end{equation*} the $@$Sangchul Lee's generalization can be proved in the same way.

Here is another solution with a generalization:

Let $r=sinhalpha$ and $s=sinhbeta$. Then $$begin{aligned} &int_{0}^{infty} arctanleft(frac{2rx}{1+x^2}right)arctanleft(frac{2sx}{1+x^2}right) , mathrm{d}x\ &= pi left( alpha sinhbeta+betasinhalpha+(coshalpha+coshbeta)logleft(frac{e^{alpha}+e^{beta}}{1+e^{alpha+beta}}right) right) end{aligned} tag{*}$$

Proof. Let $J = J(alpha,beta)$ denote the right-hand side of $text{(*)}$. Then

$$ J(0, beta) = 0, qquad J_{alpha}(alpha, 0) = 0, qquad J_{alphabeta} = pi left( frac{1+coshalphacoshbeta}{coshalpha + coshbeta} right). $$

Now let $I = I(alpha, beta)$ denote the left-hand side of $text{(*)}$. Then by the substitution $x=tan(theta/2)$, we get

$$ I = frac{1}{2}int_{0}^{pi} frac{arctan(sinhalphasintheta)arctan(sinhbetasintheta)}{1+costheta} , mathrm{d}theta. $$

From this, we easily check that $I$ also satisfies

$$ I(0, beta) = 0, qquad I_{alpha}(alpha, 0) = 0. $$

Moreover,

begin{align*} require{cancel} I_{alphabeta} &= frac{1}{2}int_{0}^{pi} frac{coshalphacoshbeta(1-costheta)}{(1+sinh^2alphasin^2theta)(1+sinh^2betasin^2theta)} , mathrm{d}theta \ &= frac{1}{2}int_{0}^{pi} frac{coshalphacoshbeta}{(1+sinh^2alphasin^2theta)(1+sinh^2betasin^2theta)} , mathrm{d}theta \ &quad - cancelto{0}{frac{1}{2}int_{0}^{pi} frac{coshalphacoshbeta}{(1+sinh^2alphasin^2theta)(1+sinh^2betasin^2theta)} , mathrm{d}sintheta}\ &= frac{1}{2}int_{0}^{infty} frac{coshalphacoshbeta (1 + t^2)}{(t^2 + cosh^2alpha)(t^2 + cosh^2beta)} , mathrm{d}t tag{$t=cottheta$} end{align*}

It is not hard to check that the last integral is equal to $J_{alphabeta}$. Therefore we get $I = J$.

$$I=int_0^infty arctan^2 left (frac{2x}{x^2 + 1} right ) dxoverset{IBP}=4int_0^infty frac{x(x^2-1)arctanleft(frac{2x}{x^2+1}right)}{x^4+6x^2+1}dx$$ We have that: $$4intfrac{x(x^2-1)}{x^4+6x^2+1}dx=(sqrt 2 +1)ln(x^2+(sqrt 2+1)^2)-(sqrt 2-1)ln(x^2+(sqrt 2-1)^2)$$ $$frac{d}{dx}arctanleft(frac{2x}{x^2+1}right)=frac12left(frac{sqrt 2+1}{x^2+(sqrt 2+1)^2}-frac{sqrt 2-1}{x^2+(sqrt 2-1)^2}right)$$ Thus integrating by parts again and simplifying we obtain: $$I=int_0^infty frac{(sqrt 2+1)^2 ln(x^2+(sqrt 2+1)^2)}{x^2+(sqrt 2+1)^2}dx+int_0^infty frac{(sqrt 2-1)^2 ln(x^2+(sqrt 2-1)^2)}{x^2+(sqrt 2-1)^2}dx$$ $$-int_0^infty frac{ln(x^2+(sqrt 2-1)^2)}{x^2+(sqrt 2+1)^2}dx-int_0^infty frac{ln(x^2+(sqrt 2+1)^2)}{x^2+(sqrt 2-1)^2)}dx$$ From here we have the following result: $$int_0^infty frac{ln(x^2+a^2)}{x^2+b^2}dx=frac{pi}{b}ln(a+b), a,b>0$$ So using this result and with some algebra everything simplifies to: $$boxed{int_0^infty arctan^2 left (frac{2x}{x^2 + 1} right ) dx=2pi ln(1+sqrt 2)-sqrt 2pi ln 2}$$