How can I use my own dataset for Image segmentation using Tensorflow

I have a huge problem using my own created dataset for image segmentation using Tensorflow. The dataset that I’ve build contain images like the one shown below:

The problem that I have is: How do I use my own dataset specifically for image segmentation? I’ve looked at the documentation on how to create datasets but all the examples either only use object detection with a single class or classify the entire image. I want to assign every pixel to a class (image segmentation) and use it to train my model.

I’ve also found examples of image segmentation but they all use existing datasets such as Cityscapes, ADE20k etc. How can I use my own images and data and transform it to a tensorflow dataset so that I can use it for training?