very hard to shift gears when clutch is activated

I have a fairly old Shimano RD-M8000 with which it becomes increasingly hard to shift gears when the clutch is activated. Particularly the mid-range gears are hard to shift into, the very low and very high gears are fairly ok. The force I have to apply is extremely high (and I figure it cannot be good for the mechanics).

What could be the reason for this? Shifter cable/tube is brand new, and shifting is smooth without the clutch activated.