Physical explanation of stress parallel to velocity gradient

For an incompressible Newtonian fluid, the stress tensor is $$sigma_{ij}=etaleft(frac{partial v_i}{partial x_j}+frac{partial v_j}{partial x_i}right).$$ Suppose only the first term is non-zero and so $$sigma_{ij}=etafrac{partial v_i}{partial x_j}.$$ This is quite easy to understand: momentum in the $i$ direction is transferred by viscous diffusion in the $j$ direction and so there is a stress in the $i$ direction on a surface with normal in the $j$ direction. By the symmetry of $sigma_{ij}$ however, we also have $$sigma_{ji}=etafrac{partial v_i}{partial x_j}.$$
How can this be explained physically in terms of molecular transport? If the velocity gradient is in the $j$ direction, how is there a stress parallel to this?