Finding $int_S z,dS$ where $S=left{ big(x,y,zbig):x^2+y^2+z^2=a^2,zge 0, a>0right}$

But you can apply the Gauss Divergence Theorem if you add to the surface the bottom disk $z=0$, $x^2+y^2le a^2$, oriented downward. You now have a closed surface and so the flux across that closed surface is the integral of $text{div},F$ over the region enclosed. As you showed, that integral will be $0$.

Therefore, it follows that the flux of $F$ across $S$ is the negative of the flux downward across the disk, i.e., the positive flux upward across the disk. (This makes sense, right? Since there are no sources in the solid hemisphere, whatever flows across the bottom must then flow outward across $S$.) But the flux of $F$ upward across the disk is easy to calculate, since $F_3=a$ is constant. We get $acdot pi a^2 = pi a^3$, as desired.

Note that Ninad's second solution is equivalent to this. But the approach using Gauss's Theorem works more generally when $text{div},Fne 0$. You just have to compute the triple integral of $text{div},F$ and then adjust by the flux across the bottom.

With the same trick, you can use Stokes' theorem in one of two ways. First is directly:

$$nabla times H = (0,0,a) to H = (0,ax,0)$$

which gives us the line integral

$$intlimits_{x^2+y^2=a^2:cap:z=0}(0,ax,0)cdot dr = a^3int_0^{2pi}cos^2 t:dt = pi a^3$$

Or we can use a corollary of Stokes' theorem, which says that we can shift the surface integral over to another surface that shares the same boundary as the original (Think distorting a bubble when you blow it from a hoop). In this case we can say that

$$iintlimits_{x^2+y^2+z^2=a^2:cap:zgeq 0}(0,0,a)cdot dS = iintlimits_{x^2+y^2leq a^2:cap :z=0}(0,0,a)cdot dS = aiintlimits_{x^2+y^2leq a^2};dA = pi a^3$$