How to calculate the force of a fan?

"Air going through the fan" means the volume of the air that passes the blades. The force that the blades exert on the air to push the air down should be the same as the gravity on the helicopter, like you said. Roughly speaking, if the helicopter has to push X air per second down to hover, and the helicopter has N blades, then every second X/N air has to be pushed down per blade. You will need a lot more information to answer this in more detail. You can't use a full circle, because then 0 air will pass the circle.

There isn't a clear equation out there, because it doesn't exist. Fluid flow is a chaotic system and the smallest details will change the outcome. Only with some gross assumptions, you can start looking at the performance characteristics of the fan blades (impeller).

The approach typically employed here is to slice each blade into a finite number of stripes, each being a 2D wing shape, with specific lift and drag characteristics as a function of incident airspeed and angle of attack (see this presentation for more details.)

Published details of of NACA 4412 airfoil

Now because of the design of the fan, the angle of attack depends on the airspeed (do a vector diagram of the velocities) and on the blade design. In the end, you will produce a single curve of axial force vs speed, as well as drag vs. speed.

Now comes the design of the motor which produces different torque at different speeds and try to match up the blade drag to the motor torque to find out how fast will the fan spin. Finally, use the speed and the curves about to get the total axial force.

---

_{An online database of airfoil characteristics.}

1. Buy a scale which can measure force in the range needed for your application.
2. Place the scale at a distance $R$ from your fan
3. Vary the wattage of your fan, and record the value on the scale for range of different wattage.
4. Perform a lagrange interpolation on the values of force per value of wattage, so now you have a function of force per unit wattage of your fan at constant distance $r_0$.
5. Set the fan to a constant wattage $w_0$, and vary the distance $r$ from the fan to the scale. Record the value on the scale for a range of distances.
6. Perform a second lagrange interpolation, now you have the force per distance of the scale at a constant wattage.
7. Where $F(r_0, w)$ is your equation for force as a function of wattage $w$ at constant $r = r_0$, and $f(r, w_0)$ is your equation for force with distance $r$ from fan at constant wattage $w_0$:

You can get the value of $f(r, w_0 + dw)$ at a different wattage like so: $$ f(r, w_0 + dw) = f(r, w_0) + frac{F(r_0, w_2) - F(r_0, w_1)}{2(w_2 - w_1)}dw $$

In this way, you don't have to learn computational fluid mechanics.

Good luck.