Is the Poynting vector always perpendicular to the plane a circuit lies in?

Given an electric circuit that lies on the plane of a sheet of paper, the magnetic field is usually "in" or "out" of the page, according to the direction of the current, and the electric field usually lies in the circuit's plane, with top-bottom or bottom-top direction (correct me if I am wrong).

I would caution you on this. It isn’t entirely wrong, but it is a pretty big approximation and simplification. So take all of the subsequent comments in that light. This is very approximate average overall behavior.

For simplicity let’s assume a single loop planar circuit with a battery on the left and a resistor on the right with the positive terminal of the battery at the top and the negative terminal at the bottom. The current will form a clockwise loop and the magnetic field of that loop will be pointing into the plane. The electric field will point in the plane from the top (positive) towards the bottom (negative).

Now, the energy flux is $vec S = vec E times vec H$ so with $vec E$ pointing down and $vec H$ pointing into the plane that means that $vec S$ points in the plane to the right. In other words, energy flows from the battery on the left to the resistor on the right, as we would expect.