How to get pixel location in after rotating the image?

You first need to define where you are rotating your image around.

If you are rotating it around (0,0) (i.e. your image is located in the positive x and y quadrant), you can apply a rotation matrix in its vanilla form:

$left[begin{array}{c}x' y' 1end{array}right] = $ $begin{bmatrix}cos theta &sin theta &0-sin theta & cos theta &0 &0 &1end{bmatrix}$ $left[begin{array}{c}x y 1end{array}right] = $ $left[begin{array}{c}xcostheta + ysintheta -xsintheta+ycostheta 1end{array}right]$

where x and y is your starting pixel location, x' and y' is your new pixel location, and $theta$ is the counterclockwise angle to rotate (you can ignore the 3rd row for now). Notice that because you are using sine and cosine functions, the new pixel location is unlikely to be an integer, so you will need to do some rounding.

However, if you are rotating around the center of the image (which is the more common use-case), you will need to first translate the "origin" of your image to the center, apply your rotation, then move your image back.

This alters the calculations to this:

$left[begin{array}{c}x' y' 1end{array}right]$

$=begin{bmatrix}1 &0 &0 &1 &0 X/2 &Y/2 &1end{bmatrix}$ $begin{bmatrix}cos theta &sin theta &0-sin theta & cos theta &0 &0 &1end{bmatrix}$ $begin{bmatrix}1 &0 &0 &1 &0 -X/2 &-Y/2 &1end{bmatrix}$ $left[begin{array}{c}x y 1end{array}right]$

$=left[begin{array}{c}(x-X/2)costheta + (y-Y/2)sintheta + X/2 -(x-X/2)sintheta+(y-Y/2)costheta + Y/2 1end{array}right]$

where capital X and Y are the size of your image in the x and y directions.

You can read more about image transformations on the mathworks page here.