Photos says some photos are in an album, but they aren't

I am trying to make a smart search that includes photos with a specific keyword and that aren’t in any folder already.

So I make a smart search:

KEYWORD is SOME_KEYWORD +
ALBUM is not ANY

I get 0 results for this, but when I remove the ALBUM is not ANY restriction, I get some 70 photos.

However these photos aren’t actually in any album. I did have them them in an album at once point, but they are no longer in one – actually, I deleted all my albums and so NONE of the photos should be in an album.

When I make a smart search for ALBUM is ANY I get 1000 results, so clearly there are some folders "in an album" – but what album? How do I remove them from the album if I don’t actually have any albums?