IOU accounting for the difference of the damage degree in GT and prediction

I have a model of a skin disease condition. It takes a skin image and predicts areas affected by edema. Skin can be in one of four degrees, so each pixel is assigned a value. Healthy: 0, mild: 1, moderate: 2, severe: 3. I want to compare ground truth mask and predicted mask. Would there be only two degrees (healthy vs disease), I would use IOU. However, I want to account for the difference of the GT and predicted damage degrees. If the IOU value is the same the metrics should be higher if GT and predicted degree matches and lower if GT predicts mild degree, while the model predicts severe damage. What is the right way to account for IOU and the difference in damage degrees?