CNN 3D line angles prediction regression - results of training of phi depend on theta

I am a beginner in “deep learning”. What I am trying to do, is to predict two angles of a 3D line projected on a 2D image. The toy model is that I create a line going out from the centre of 48×48 array. The angle phi is the angle of the line in the image plane, angle theta is the angle of the line in the plane perpendicular to the image. Theta is just used to calculate each line point distance from the image centre and thus the line brightness at each point, by scaling line value with 1/R^2.

The problem is, that with theta fixed to 0 I can achieve ~3 deg precision of phi determination. However, if theta is allowed to go between 0 and 90 deg, the phi training results drop significantly. Of course, prediction of phi is worse for theta approaching 90 deg, because for theta=90 deg the phi is undefined. However, the results of phi training depend on the theta range allowed. I will have much better results of phi training if I allow theta in the range of 0-45, than if I allow theta in the range of 0-90. I would be grateful if someone could tell me, why this is the case and how to improve the results.

As for implementation. I am using Chainer based CNN. What I do with my layers does not seem to affect results much. At the moment I have 6 2D convolutional layers then 2 linear layers, the last one giving 3 variables. The variables are sin(phi), cos(phi), theta. Using sin/cos it is easier to cover whole 360 deg and take into account circular nature of angle. I output variable as is (linear). The loss function calculates phi as arctan2(cos(phi),sin(phi)), then uses mean_squared_error to compare its phi,theta with the input.