Magnetic circuit: What is the cross section area I need to use?

The reluctance of the core has nothing to do with $A_g$, so $$R_c = frac{l_c}{mu A_c}. $$

The two overlap of the core and the plunger is $A_g$, so you would write $$R_g = frac{g}{mu_0 A_g}$$ The right value would probably be some average of $A_c$ and $A_g$.

Note that the reluctances of the plunger and the gap are in parallel, so you would write $$ R_p = left[left(frac{l_p}{mu A_p}right)^{-1}+ left(frac{l_p}{mu_0 (A_c - A_g)} right)^{-1} right]^{-1} $$ where $A_p$ is the area of the plunger. If $mu gg mu_0$ and the plunger isn't too far out, this can be simplified to $$ R_p = frac{l_p}{mu A_p}. $$

The truth is the correct areas to use aren't always obvious: simple models like this are useful for quick hand calculations but don't always predict the actual results very accurately. For example, in this case the reluctance $frac{l_p}{mu A_p}$ of the plunger may not be accurate if the plunger is very thin: then the correct area would be some average of $A_p$ and $A_g$. Note that this analysis neglects also neglects fringe fields and uses a rather simple equation for the core reluctance, so don't expect it to match up exactly with your simulation results.