What has more statistical power when determining glm parameter importance, comparing models with a dropped parameter or coefficient pvalues?

Question

I am looking to determine the significant of a parameter in a negative binomial distributed glm to determine if origin (either isolate or free) is important in the model:
mnegbin1 = glm.nb(count ~ origin + substrate, data = some_data)
summary(mnegbin1)

Coefficients:
                                     Estimate Std. Error z value Pr(>|z|)    
(Intercept)                         -2.194329   0.588844  -3.727 0.000194 ***
originisolate                       -0.119740   0.071953  -1.664 0.096084 .  
substrateagarose                    -1.099756   1.164682  -0.944 0.345040    
substratealcohol                    -0.408900   0.926243  -0.441 0.658880    
substratealginate                    1.201032   0.676161   1.776 0.075691 .  
substratealpha-glucan                3.903481   0.603129   6.472 9.67e-11 ***

Is is more powerful to just look at the pvalue of the originisolate coefficient given that origin is either isolate or free OR is it better to compare the residual deviance of this model to one dropping origin as a parameter?
mnegbin2 = glm.nb(count ~ substrate, data = cazy_glm)
anova(mnegbin1, mnegbin2, test = "Chisq")
Likelihood ratio tests of Negative Binomial Models

Response: count
               Model    theta Resid. df    2 x log-lik.   Test    df LR stat.    Pr(Chi)
1          substrate 2.826295      1260       -2752.491                                 
2 origin + substrate 2.845114      1259       -2749.730 1 vs 2     1 2.760444 0.09662139

Thomas Lumley · Answer

An important basic fact about parametric likelihood estimation is that the Wald, score, and likelihood ratio tests are locally asymptotically equivalent. What that means is that in settings where you have some ability to reject the null hypothesis but it's not completely obvious, the three tests will give similar answers. That fits what you see in your example.
There isn't any general result (as far as I know) about which one is slightly better in those circumstances, but the statistical folklore says that the Wald test (the $p$-value of a coefficient, as you put it) is less reliable than the other two. Differences in power between the three tests are as likely to be due to differences in the accuracy of the Normal approximation as to differences in the true power.

What has more statistical power when determining glm parameter importance, comparing models with a dropped parameter or coefficient pvalues?

One Answer

Add your own answers!

Ask a Question