When NOT to create spatial index

As I realized once again connected with this answer to Joining attributes by location using QGIS, creating a spatial index can result in huge performance improvements.

However, I was wondering why spatial indexes are not created as a default in GIS software.

Are there any reasons why one should NOT create a spatial index?

I use QGIS 3.16 on Win10 and I am looking for answers for this setting, but I guess the answer will be not software-specific.

The only possible reason I could imagine not to create a spatial index is again connected with performance: creating a spatial index takes some time/processing power and storage capacity. For smaller datasets, these indexes do not bring such a huge improvement. And if you only want to visualize data (map making), spatial indexes are of no use (though maybe even that is not the case: https://gis.stackexchange.com/a/62302/88814).

Still, considering the huge performance improvement related to the modest additional requirements for processing and storing, I see no reason why one should not always create a spatial index.

Are there reasons/scenarios not to do so?