What are some existing techniques for pose estimation angle normalization?

So I am currently building a model which does a certain type of action recognition, which I am implementing as a two-stage, end-to-end system. The first stage is a pose estimation model, and I want to take the outputs from this model and use some type of sequence classification model (to be determined what architecture specifically, not important for question) for outputting the type of action. However, this is essentially based on the changes in the relative positions of different segments from the pose estimation model, which are variable to the angle that the input video was shot at. So, as I see it, I will need to either gather a much larger amount of data to generalize across angles, make sure every input video is from roughly the same angle, or normalize the pose estimation data by essentially rotating it.

The way I’m thinking about this is I could

1. train another network which could rotate the input video beforehand (there is some research on this, and I could potentially generate some of the training data myself by using perspective transforms), or

2. figure out some type of mathematical model which will give me the angle based on segment relationships to either train data for (1) or to derive some sort of transform based on this model to convert the estimated segment positions to a standard view

---

I was wondering if there is any other existing techniques I haven’t been able to think of or anything like that which would help me out. Any help greatly appreciated, thanks!