TransWikia.com

Is it ok to trust regression predictions when none of the coefficients are statistically significant?

Data Science Asked on December 30, 2021

Background to the problem: I am estimating individual treatment effects using double machine learning model. I do not know true treatment effects for my problem.

Double ML: Given Y (outcome), T (treatment) and X ( features)

Y = aT + bX + error

coefficient a is of interest (measures treatment effect) .

Double ML procedure:

  1. Fit Y ~ X => Compute residuals (Y* = Y – Y’) – Residuals are
    treatment effects to be estimated
  2. Fit T ~ X => Compute residuals (T* = T- T’) – This model captures
    variation in T explained by X
  3. Fitting a model (Y* ~ T* ) on residuals will give the average
    treatment effects

I am fitting a linear regression model (Y* ~ T* ) and none of the coefficients are statistically significant. Instead of relying on point estimates, I am computing prediction confidence intervals and p-value to check if the predicted value is statistically significant or not.

Is this approach good?

One Answer

It depends on the goal of the project.

Statistical significance is more important for publishing in academic journal. For applied projects, statistical significance is less important.

Even if a model is not statistical significant, it can have value. The model may summarize the data and identify relationships. The model maybe practically important, the effect maybe of value to practitioners.

Answered by Brian Spiering on December 30, 2021

Add your own answers!

Ask a Question

Get help from others!

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP