Image segmentation with large class imbalance leads to zero precision/recall

I have a binary semantic image classification problem where only very small parts of the images are positive, most of it is negative. In the training data I have a positive rate of around 0.023, which is a large imbalance (factor 43). Now when I train, the output is just that every single pixel is negative, which leads to a high accuracy. However, it does not locate the particular symbols that I want to find on the pictures. Increasing the training data won’t remove this imbalance because the glyph I want to find covers less than half of the bounding box, therefore even an image full of that glyph would be imbalanced.

I’ve read that one can set weights for the categories, but Keras Model.fit does not support class_weight when the output has three or more dimensions (samples, pixels, classes) but only when classifying samples as a whole (samples, classes).

Is there a way to reduce the false-negatives and increase the true-positives?