CNN for Coordinates Prediction

I am working on a CNN to detect body parts (Right Shoulder, Left Shoulder), something more like Facial Keypoints Detection. I tried solutions from Kaggle as I think it’s almost the same problem but it did not work very well.
More details:

• I collected and labeled my own data,
• I am using generators to augment the images by only resizing the image and place it in 90x30px frame.
• I normalized the input images and the output arrays to get values between 0 and 1.
• I tried MSE loss and a RMSE with different CNN architectures.

Full notebook available here
[Note] I have bigger dataset than the one shown in notebook, it’s 10K samples
so could it be the data? the architecture? or simply the optimization setup?