In tetrahedron ABCD, prove that $r<frac{ABcdot CD}{2AB+2CD}$

Let $MN$ be the common perpendicular line of $AB$ and $CD$ with $M$ a point in $AB$ and $N$ in $CD$. The volume $V$ of the tetrahedron $ABCD$ can be calculated in two ways:

$$begin{align} V &= frac16 times AB times CD times MN times sin(alpha) \ &= frac13 times r times (S_{triangle ABC} + S_{triangle BCD} + S_{triangle CDA} + S_{triangle DAB}) end{align}$$ where $alpha$ is the angle between $AB$ and $CD$.

Notice that $$ S_{triangle ABC} = frac12 times AB times MC > frac12 times AB times MN, $$ and this is true for all other $3$ triangles. Thus $$begin{align} V &= frac13 times r times (S_{triangle ABC} + S_{triangle BCD} + S_{triangle CDA} + S_{triangle DAB}) \ &> frac16 times r times (AB times MN + CD times MN + CD times MN + AB times MN) \ &= frac13 times r times MN times (AB + CD) end{align}$$ It follows that $$begin{align} r times MN times (AB + CD) &< 3V \ &= frac12 times AB times CD times MN times sin(alpha) \ &leqslant frac12 times AB times CD times MN end{align}$$ So finally $$ r < frac{AB times CD}{2(AB + CD)}.$$

It is clear when $vec{AB} parallel vec{CD}$, the tetrahedron $ABCD$ is degenerate and $r = 0$, the inequality will be trivially true. Let us consider the case $vec{AB} notparallel vec{CD}$. Under this assumption, one can choose a coordinate system such that

• the incenter is centered at origin.
• $AB$ is lying in the plane $z = u > 0$.
• $CD$ is lying in the plane $z = -v < 0$.

Let $p = |AB|$, $q = |CD|$ and $displaystyle;tau = frac{v}{u+v} in (0,1)$.

Let $E, F, G, H$ be the intersection of the edge $AC$, $BC$, $AD$, $BD$ with the plane $z = 0$. The intersection of the tetrahedron $ABCD$ with the plane $z = 0$ will be a quadrilateral with these 4 points as vertices. It is easy to see

• $vec{EF} parallel vec{GH} parallel vec{AB}$ and $|EF| = |GH| = ptau$.
• $vec{EG} parallel vec{FH} parallel vec{CD}$ and $|EG| = |FH| = q(1-tau)$.

The quadrilateral is actually a parallelogram with sides $ptau$ and $q(1-tau)$.

The intersection of the insphere with the same plane will be a circle of radius $r$. It is clear this circle lies inside above parallelogram.

Since the distance between the line $EF$ and $GH$ is at most $q(1-tau)$ and the distance between the line $EG$ and $FH$ is at most $ptau$, we have

$$2r le min( ptau, q(1-tau) )$$

Treat the RHS as a function of $tau in [0,1]$, it is maximized when $$ptau = q(1-tau) iff tau = frac{q}{p+q}$$ This give us

$$2r le max_{tin[0,1]}min(p t, q(1-t)) = frac{pq}{p+q} quadimpliesquad 2r le frac{|AB||CD|}{|AB|+|CD|}tag{*1}$$

This is pretty close to what we want to prove. To show above inequality is actually strict, we need two observations.

1. The insphere touches the surface of the tetrahedron at 4 points, one at each face. Furthermore, the corresponding face is perpendicular to the radial direction there.
2. In order for the equality to $(*1)$ to hold, we need $ptau = q(1-tau) = 2r$. On the plane $z = 0$, the parallelogram is actually a square and the circle of radius $r$ need to touch the 4 sides of it.

Combine these two observations, we find in order for the equality to hold, the face holding $EF$ need to parallel to that holding $GH$ and the face holding $EG$ need to parallel to that holding $FH$. Their normal vectors are all orthogonal to the $z$-direction. These four faces are now bounding an infinite long cylinder instead of a tetrahedron. This is absurd and we can conclude

$$2r < frac{|AB||CD|}{|AB|+|CD|}$$

Update

At the end is a picture which hope to illustrate the configuration. The vertices of the tetrahedron is located at

$$A,B,C,D = (2,0,2), (-2,0,2), (2,-2,-2), (-2,2,-2)$$

Both $vec{AB}$ and $vec{CD}$ are lying in some planes perpendicular to $z$-axis. The insphere is centered at $left(0,0,2left(frac{3-sqrt{5}}{3+sqrt{5}}right)right)$ with radius $r = frac{4}{3-sqrt{5}}$. If one cut the tetrahedron with the plane $z = 2left(frac{3-sqrt{5}}{3+sqrt{5}}right)$ which passes through the incenter, one obtain a parallelogram $EGHF$ ( $F$ is behind the insphere and not visible from this viewpoint ). The intersection of the insphere with that plane is a circle of radius $r$ which is contained within the parallelogram $EGHF$.

$hspace 0.75in$