What do force extension graphs for springs show?

The force vs extension graph which is drawn is not the net force on the spring rather it is either the external force acting on the spring (F = kx) or the force exerted by the spring (F = -kx) where x is the extension of the spring.

You could imagine a situation when the external force is increase from zero to 4 N in such a way that the resultant force on the spring is always very small and in the limit infinitesimally small.

You may think of spring force to be a self adjusting force similar to static friction except that there is no upper bound. Since one end of the spring is attached to a rigid body, the net force will have to be zero as the spring cannot move as a whole. Hence the spring force is given as a function of extension as $F=-kx$

You have a point, but it comes from a misunderstanding. In $F = k x$, the $x$ stands for deflection and not length.

Typical spring performance graphs show force vs. length, where the free length is evident, as well as the intended working range.