Remove Locator doesn't work

I have the issue that I can’t remove Locators, when using LocatorAutoCreate->True.
For example the code snippet taken from "LocatorPlane" -> Options-> LocatorAutoCreate

DynamicModule[{pts = {{-1, 1}/2, {1, 1}/2}},
 LocatorPane[Dynamic[pts], Framed@Graphics[{}], LocatorAutoCreate -> All]]

As documented I can add additional Locators by clicking an empty space. But the removal by Alt+Click doesn’t work. Instead it moves the Locator by a very small amout, if I drag the mouse around (as if the mouse movement is scaled down).

In the past I had the issue, that Xubuntu was intercepting Alt+Click with some OS related feature. I removed that keybinding and it was working fine in the past (>1 years ago). But right know, I have the same issue again.

Currently I use the latest Mathematica version 1.12.1 and Xubuntu 20.04, but this problem did occur in previous versions of Mathematica and the previous LTS of Ubuntu.
This issue doesn’t occur on Windows. I don’t know about the Mac version.