Continuity equation, why doesn't it apply here?

The correct way to solve the problem is to use Bernoulli's principle along with the following constraint: for the siphon to work, the pressure must be positive throughout the system. Using Bernoulli's principle (with the idealization that the velocity is zero at the surface of the reservoir), we get

$$ frac{p_{textrm{atm}}}{rho} = frac{p_{textrm{peak}}}{rho} + frac{v^2}{2} + gH $$

where $ v $ is the flow velocity inside the tube. $ H $ will be maximized in this equation if $ p_{textrm{peak}} = 0 $ and $ v = 0 $, so the maximum possible $ H $ is

$$ H_{textrm{max}} = frac{p_{textrm{atm}}}{rho g} $$

As for your question about the continuity equation, it does apply here: the flow velocity $ v $ is constant throughout the entire tube. However, in this application of Bernoulli's principle we are comparing the velocity at the surface of the reservoir with the velocity inside the tube, which aren't equal and aren't required to be by the continuity equation.