QGIS: data defined position of bar diagram

I am using QGIS to visualise communal water balances of different years as bar plots (QGIS 3.8). I have a point layer (communal area’s centroids) with the balance values as attributes.
I’d like to show deficits in reds and surpluses in blue. Given that QGIS only supports one colour per bar, I decided to do create two plots at the same location. As stated here, bar plots do not support all-negative values. Thus, I inverted the bar orientation of the deficit graph and print only negative values
[*if(value<0,-value,0*].

Now, the bar plots seem to be oriented in the middle quadrant. As I want both graphs to share y=0, I moved the negative plot by setting the x-Coordinate to $x and the y Coordinate to $y-(max(values) -min(values))/(bar length per map unit)/2. This led to the expected bar plot:

However, the bar graph’s position is not fixed on the map, but moves around when I change the map extent (as if the placement setting were set to "around point"). In order to label the bars, I need them to remain at ($x,$y) strictly.