Fire an alert when number of sign up in an app drops. How to find the best condition to maximize accuracy?

First, you need to think about what you mean by accuracy. Do you want to minimise false alarms (false positives)? Minimise the number of drops that you miss (false negatives)? Some combination of the two? Many textbooks provide good discussions of this , but Wikipedia is also a good place to start.

Now, to the question. To control the false positive rate, you can calculate the one-tailed p-value: the probability of observing a conversion rate as low as that in the previous hour or lower, under the null hypothesis that there's nothing wrong with the app. In your case, this p-value can be calculated using a binomial test, where the null hypothesis is that $P(text{Convert}) = 0.4$. Importantly, this test deals with with the fact that the app open count fluctuates from hour to hour, and that the conversion rate is more variable at times when fewer people open the app.

If you set the alert to fire when $p < .05$, or $frac{1}{20}$, and run the test every hour, you will on average have one false positive every 20 hours when the app is working fine. A lower threshold (e.g. $p < frac{1}{100}$) will lead to fewer false positives (one every 100 hours), but also makes it harder to detect real problems.

How likely are you to detect problems when they do arise? As your tags suggest, this is called the power of the test. This depends on a) your p-value threshold (call this $alpha$), b) how low the conversion rate goes when problems occur, and c) how many people open the app each hour.

This calculator gives you power for a binomial test given all of these values. The screenshot below shows a calculation assuming that a) we fire an alert if $p < .05$, b) the conversion rate goes down to 30% when there is an issue, and c) 200 people opened the app in that hour.

Finally, you can always reduce both your false positive and your false negative rates by simply using a longer window. If you check every two hours, rather than every hour, keeping $alpha = 0.05$, you will have a false positive on average once every 40 hours (20 tests), and since you're including more data in each test, the power of the test (the ability to avoid false negatives) will be greater.

As far as I understand it, you are dealing with a binomial distribution, with $p approx 0.4$ being the typical (normal) conversionRate, the number of trials $n = $ appOpenedCount and the number of successes $k = $ signupCount.

There is, however, a piece of information missing in your question. You say:

Usually, we want this to be in the past 1 hour to be urgent.

but don't provide the app open rate (as per hour). If this rate is very low, say, one app open event per month, you can't give the alert within your specified time frame, ever.

What you can do is decide on the optimal number of app open events to detect a drop in the conversion rate with some pre-defined power, say, $0.9$. In R multiple libraries seem to be capable of doing power analysis for binomial distribution, and this page explains how to do it in Excel.