Generation of 'new log probabilities' in continuous action space PPO

I have a conceptual question for you all that hopefully I can convey clearly. I am building an RL agent in Keras using continuous PPO to control a laser attached to a pan/tilt turret for target tracking. My question is how the new policy gets updated. My current implementation is as follows

1. Make observation (distance from laser to target in pan and tilt)
2. Pass observation to actor network which outputs a mean (std for now is fixed)
3. I sample from a gaussian with the mean output from step 2
4. Apply the command and observe the reward (1/L2 distance to target)
5. collect N steps of experience, compute advantage and old log probabilities,
6. train actor and critic

My question is this. I have my old log probabilities (probabilities of the actions taken given the means generated by the actor network), but I dont understand how the new probabilities are generated. At the onset of the very first minibatch my new policy is identical to my old policy as they are the same neural net. Given that in the model.fit function I am passing the same set of observations to generate ‘y_pred’ values, and I am passing in the actual actions taken as my ‘y_true’ values, the new policy should generate the exact same log probabilities as my old one. The only (slight) variation that makes the network update is from the entropy bonus, but my score
np.exp(new_log_probs-old.log_probs) is nearly identically 1 because the policies are the same.

Should I be using a pair of networks similar to DDQN so there are some initial differences in the policies between the one used to generate the data and the one used for training?