Why don't my tracking markers stay in position?

This is my very first post here. After having used it a few times in the past, I recently started playing around with Blender again. At the moment I’m experimenting with Blender’s camera tracking feature, and I keep running into an issue that I’m unable to solve.

The problem is that the markers of tracks don’t stay in the position I put them in.
For example, when I place the marker in the first few frames of my video footage (either manually or by clicking the Track Markers button for automatically tracking a single frame) everything looks fine when I play the first few seconds of the video: the marker perfectly follows the item in the footage that I’m trying to track.
But when I continue to work on the same track, and move the marker in another part of the video (for example, around frame 532), and then go back to the first few frames, the marker has shifted to a different position there.

Example:
I manually placed the marker in the right position in frame 17:

After manually moving the marker in a few frames around frame 532, the marker in frame 17 looks like this:

NOTE: In this example, I didn’t press any Tracking button. All the changes to the marker position were done manually.

How is this possible? Is this a bug in Blender, or did I accidentally activate a setting that causes changes in frame x to influence the position of the marker in frame y?

I’m using Blender 2.90.1, and my tracking settings look like this:

Any help would be much appreciated!

Kind regards,

Frank