# Relationship between projection of $y$ onto $x_1, x_2$ individually vs. projection on both?

Mathematics Asked by roulette01 on September 29, 2020

This is essentially similar to the question I just asked on cross validated, but here I am going to pose it in a linear algebra way.

Consider $$y in mathbb{R}^n$$ and $$x_1, x_2, 1_n in mathbb{R}^{n}$$. Suppose you orthogonally project $$y$$ onto $$x_1, 1_n$$ and find the projection of $$y$$ onto the subspace spanned by $$x_1, 1_n$$ can be written as $$hat{y}_1 = hat{beta}_1 x_1 + b_1$$, i.e., a linear combination of $$x_1$$ plus some offset. Now do the same for orthogonal projection of $$y$$ onto $$x_2, 1_n$$ and find $$hat{y}_2 = hat{beta}_2 x_2 + b_2$$.

Now consider projecting $$y$$ onto the subspace spanned by both $$x_1, x_2, 1_n$$ and find $$hat{y}_{12} = hat{gamma}_1 x_1 + hat{gamma}_2x_2 + b_{12}$$.

If $$x_1 perp x_2$$, then I know $$hat{beta}_i = hat{gamma}_i$$. But what if they’re not orthogonal?

What can I say about the relationship between $$hat{beta}$$ and $$hat{gamma}$$ in this case?

Some specific questions that I am also interested in is if $$hat{beta} >0$$, does this imply $$hat{gamma} > 0$$? If $$x_1, x_2$$ are linearly dependent, then I don’t think this won’t be true for one of the coefficients.