Aren't measurements in the Stern Gerlach experiment inherently intrusive to the states of particles?

Your story is not completely correct. Let's remember about Larmor precession classically. It says that if you have a magnetic moment vector $vec{mu}$ in an external magnetic field $vec{B}$, that $vec{mu}$ will rotate around $vec{B}$ with a constant frequency. If a $vec{mu}$ is aligned or anti-aligned with $vec{B}$, then there will be no precession. If $vec{B}$ is constant in space then there is no net force on the particle.

If $vec{B}$ is changing in space, then another effect kicks in. Namely, there is a net force of $$ vec{F} = -vec{nabla} (vec{mu} cdot vec{B}). $$ In the Stern Gerlach apparatus, there is indeed a magnetic field which is changing in space. If the particle is spin up, and $vec{mu}$ is aligned with $vec{B}$, then the force accelerated the a particle in the direction of decreasing $vec{B}$. If the particle is spin down, it is accelerated the other way. This is how the beams are separated depending on if they are spin up or spin down.

Note that Larmor precession actually does not play a role here. If the spin is completely aligned or anti aligned with the field, it plays no role. Because no actual Larmor precession is occuring in this experiment, it is not fair to say it is the reason that the information about the original state is lost is that the Larmor precession somehow washes it out.

Edit: Quantum spin should not be thought of as a three dimensional vector, like $(v_x, v_y, v_z)$. Instead it is described by a two dimensional vector with complex components. In particular, $$ | uparrow_x rangle = frac{1}{sqrt{2}}begin{pmatrix} 1 1end{pmatrix} hspace{1 cm}| downarrow_x rangle = frac{1}{sqrt{2}}begin{pmatrix} 1 -1end{pmatrix} $$ $$ | uparrow_z rangle = begin{pmatrix} 1 0end{pmatrix} hspace{1 cm}| downarrow_z rangle = begin{pmatrix} 0 1end{pmatrix}. $$ Note that, while in three dimensions the vector $(1, 0, 0)$ is orthogonal to $(0, 0, 1)$, when it comes to the quantum state we have $$ | uparrow_x rangle = frac{1}{sqrt{2}} | uparrow_z rangle + frac{1}{sqrt{2}} | downarrow_z rangle $$ so the up state in the $x$ direction is one half the up state in the $z$ direction and the down state in the $z$ direction. Thus if you measure the spin of $|uparrow_xrangle$ in the $z$ direction, one half of the time you find it spin up and the other half of the time you find it spin down.