What's the difference between "family affair" and "family stuff"

Stuff is very informal. You wouldn't use to to speak to the Queen.

Also, it's an all encompassing term, it could describe a party, a funeral, an intervention, etc. Whereas you probably wouldn't refer to a family funeral as a 'family affair'.

For me the difference between them is "stuff" you can do it anytime while "family affair" comes only sometimes or once a while.