I don't understand the discontinuity in electric field across a surface

That formula refers to what happens when the surface charges are immersed in an external electric field and you wish to know what happens to the $total$ electric field at the boundary. You have a field from the surface charges $pm E^s =frac{sigma}{2epsilon_0 }$ where $pm$ shows whether "below" or "above", and let an external field $E^x$ be perpendicular to the plane of the surface charges. Then on one side of the plane $E^x-E^s$ while on the other side you have $E^x+E^s$ and the difference of the two, i.e., the jump discontinuity across the insulator of the normal component of the total $mathbf{E}$ field is exactly $2E^s=sigma/epsilon_0$.

So won't that mean that the perpendicular component of the electric field above and below the surface would get added up instead of being subtracted?

No. Let the surface charge be located at $z=0$, the area of the top and bottom surfaces of the Gaussian pillbox be $A$, and the electric field immediately above and below the surface be $mathbf E_1$ and $mathbf E_2$. In the limit as the vertical height of the box goes to zero, the flux through the side walls goes to zero, so the total outward flux is $$Phi_E simeq mathbf E_1 cdot A hat z + mathbf E_2 cdot A(-hat z) = (E_{1z} - E_{2z})A$$

which is equal to $sigma A/epsilon_0$ by Gauss' law. Therefore,

$$E_{1z} - E_{2z} = sigma/epsilon_0$$

---

Isn't this the same way we get that the electric field for an infinite charged plane is sigma/2*epsilon?

No. This derivation goes as follows. First, we assume translational and rotational symmetry in the $x$ and $y$ directions - this implies in particular that the electric field only has a $z$ component. Next, we assume reflection symmetry across the $z=0$ plane. This implies that the electric field above the $z=0$ plane is the mirror image of the electric field below it.

Putting this together, we can make a Gaussian pillbox which is not infinitesimally thin in the vertical direction, as long as it extends the same distance above $z=0$ as below it. Because the electric field is purely vertical, the only flux is through the top and bottom surfaces. Because of the inversion symmetry, they contribute precisely the same amount - $Phi_E = EA + (-E)(-A) = 2EA$. From here, Gauss' law gives that

$$2EA = sigma A/epsilon_0 implies E = sigma/2epsilon_0$$

---

These two derivations differ because in the latter case, we assume (i) that the electric field is vertical, (ii) that the electric field is invariant under translations and rotations in the $xy$ plane, and (iii) that the electric field has mirror symmetry across $z=0$. In a general case, none of these things will be true, so the best we can do is demonstrate how the perpendicular componennt of the electric field jumps as you cross $z=0$. Note that if you assume mirror symmetry (so $E_{2z} = -E_{1z}$), you recover the familiar result for an infinite plane of uniform charge density.