TransWikia.com

Does a significant interaction necessarily implies that at least one of the simple effects will be significant?

Cross Validated Asked on November 2, 2021

Considering that the interaction between two variables is significant, is it always the case that at least one of the two simple effects will be significant? If so, is there any proof or a counter example?

For the sake of simplicity, let’s assume that we are talking here about 2 x 2 or 2 x continuous designs.


Edit

Example:

set.seed(1)
n <- 100
x    <- rep(c("a", "b"), each = n/2)
x.ab <- ifelse(x == "a", -.5, +.5)
x.a  <- ifelse(x == "a",  0, +1)
x.b  <- ifelse(x == "b",  0, -1)
z    <- rnorm(n)
y    <- x.ab*z + rnorm(n)

We now test the interaction:

> summary(lm(y ~ x.ab*z))

             Estimate Std. Error t value Pr(>|t|)    
(Intercept) -0.037776   0.097254  -0.388    0.699    
x.ab         0.235133   0.194508   1.209    0.230    
z            0.001777   0.109290   0.016    0.987    
x.ab:z       0.946657   0.218581   4.331 3.65e-05 ***

The x.ab:z interaction is significant (and there are no main effects of neither x nor z), so we can test for simple effects:

> summary(lm(y ~ x.a*z))

            Estimate Std. Error t value Pr(>|t|)    
(Intercept)  -0.1553     0.1375  -1.129  0.26151    
x.a           0.2351     0.1945   1.209  0.22969    
z            -0.4716     0.1659  -2.843  0.00546 ** 
x.a:z         0.9467     0.2186   4.331 3.65e-05 ***

The simple effect y ~ z is significant when x = a (p = 0.005).

> summary(lm(y ~ x.b*z))

            Estimate Std. Error t value Pr(>|t|)    
(Intercept)  0.07979    0.13754   0.580   0.5632    
x.b          0.23513    0.19451   1.209   0.2297    
z            0.47510    0.14235   3.338   0.0012 ** 
x.b:z        0.94666    0.21858   4.331 3.65e-05 ***

The simple effect y ~ z is also significant when x = b (p = 0.001).

As we can see in this example, both simple effects are significant. Is it always the case that at least one of the simple effects will be significant, given that the interaction is significant?

Ps: What I refer to as "main effects" and "simple effects" is based on Spiller et al. (2013).

One Answer

Considering that the interaction between two variables is significant, is it always the case that at least one of the two simple effects will be significant?

No, not at all.

If so, is there any proof or a counter example?

Yes, it is easy to create a counter example:

> set.seed(1)
> N <- 200

> A <- rep(c(0,0,1,1), times = N/4)
> B <- rep(c(0,1), times = N/2)
> y <- A*B + rnorm(N)

> summary(lm(y ~ A*B))

Coefficients:
            Estimate Std. Error t value Pr(>|t|)   
(Intercept)  -0.0407     0.1317   -0.31   0.7579   
A             0.2250     0.1863    1.21   0.2287   
B             0.0789     0.1863    0.42   0.6723   
A:B           0.6971     0.2635    2.65   0.0088 **

I would always advise against relying on statistical significance, which relies on p-values (that depend on sample size) and arbitrary significance levels.


Edit:

Following the update, I have to say that I hope the procedure in the edited question is not something that is being used in practice, but anyway, yes of course we can find a counterexample. Just set the seed in your code to, for example, 276

Answered by Robert Long on November 2, 2021

Add your own answers!

Ask a Question

Get help from others!

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP