Having difficulties deriving the formula for the force acting upon a dam with height $H$ and width $L$

So you can actually do your integration much easier: $$ int_0^hmathrm dz~frac{mathrm df}{mathrm dz}=f(h)-f(0) $$ and for your expression $f(z) = z~ p(z)$ satisfying $f(0)=f(h)=0$, the former from the leading $z$ and the latter from the fact that $p(h)= 0.$ So you implicitly solved the integral when you wrote down this expression for $F(z)$.

A further hint for why this expression is unsatisfying is that it does not seem to play well with an arbitrary origin. Suppose we instead defined $z=0$ to be the top of the dam and had $int_{-h}^0mathrm dz$, I don't see where your argument would not also yield $frac{mathrm dphantom z}{mathrm dz}(z~p(h+z))$ for this integrand, but now you have a completely different integral yielding $h p(0)$ at its lowest.

As for what went wrong in this derivation, I am tempted to say it was a failure of a sort of “magical thinking” or so? There is no reason to believe that $mathrm d(z~p(z))$ has anything to do with the resulting force. So we should take a step back and think more carefully.

What is at stake here is the physical meaning of the underlying Riemann sum. If we get the Riemann sum right, then we will get the right integral.

Draw two actual points in the middle of the dam, $z_k,~z_{k+1}$, ask what the force is on the part of the dam between these two points. You have an average pressure $bar p_k = frac12big(p(z_k)+p(z_{k+1})big),$ and you have a surface area $L~(z_{k+1}-z_k)$ so the right expression is the Riemann sum $$ F=sum_{k=0}^{N-1} bar p_k ~L~(z_{k+1}-z_k). $$ In a fine mesh we know $bar p_k$ will be approximately $p(z_k)$ and if you are not yet fully experienced you may prefer to take this out to an extra order, $$ bar p_kapprox p(z_k) + p'(z_k) frac{z_{k+1} - z_k}2,$$ but you won't have nonzero contributions from $( mathrm dz)^2$ until your stochastic calculus course, which will do calculus with non-differentiable functions. Here, the $p'$ term eventually vanishes and you get your book’s expression $p(z)~L~mathrm dz$ as the appropriate integrand. If you prefer, you might write $p~mathrm dA$ with $mathrm dA=L~mathrm dz$ to make this look more like a pressure times a little area, summed up over the area.

Of course, calculus is overkill for this problem, as this is a straightforward case of a linear increase in pressure from 0 to a maximum at the bottom of the dam, the average pressure is therefore the pressure at the middle of the dam, answer $frac12 rho~g~H cdot HL.$ A more interesting problem: very often the dam is smaller on the bottom than it is on the top, to first approximation maybe we assume that it looks like a parabola $ell(z) = L~sqrt{z/H}$, confirm that the proper Riemann sum gives $int_0^H mathrm dz ~p(z)~ell(z)$ and solve that.

Your equation $$dF=p(z)L dz$$ correctly describes the force ($dF$) acting on an area element (width $L$, height $dz$) of the dam.

However, your equation $$dF=frac{partial (p(z)cdot Lz)}{partial z}dz$$ is not motivated by anything physical. Therefore all conclusions derived from that are unfounded.

Pressure is defined as $$P=frac{F}{A}.$$ But this formula assumes that the force, $F$, is constant over a given area, A. If this is not the case then one makes the assumption that for an infinitesimally small area dA the force dF is constant, hence the formula turns into $$P=frac{dF}{dA}.$$ This, for your question can be written as $$dF=PdA$$ $$int_{area}dF=int_{area}PdA$$ $$F_{total}=Lint Pdz.$$ Now let me explain your mistake with total derivative $dF$. Force is a function of pressure and area, that is, $$F=Fleft (P,A right ).$$ Therefore the total derivative is $$dF=frac{partial F}{partial P}dP+frac{partial F}{partial A}dA$$ that is $$dF=AdP+PdA.$$

If you want to integrate both sites you just end up with a formula similar to integration by parts $$F=int AdP+int PdA$$ or $$int AdP=F-int PdA.$$