Is direct, complete conversion of matter to energy theoretically possible?

I know that small amounts of matter can be converted to energy via chemical and nuclear reactions, and that complete conversion is possible if matter meets anti-matter. Other types of conversion might proceed more exotically, e.g., in a black hole collision. But I have a more theoretical question; is there any reason in principle that there couldn’t be some way to directly turn 100% of a normal piece of baryonic matter into pure energy, without the trouble of first making anti-matter? (And, ideally, without most of it coming in the form of gamma rays, though with a large enough reactor one could convert this to heat and hence useful energy.) Obviously we don’t know any practical way to do this–we’re not even at fusion yet–I’m just wondering if there’s some fundamental barrier to it.

Part of my interest is speculation on the Fermi problem; if large civilizations even in distant galaxies used the most readily-available source of energy we are aware of (stars) via, say, dyson swarms, we would notice a spectrum change tipping us off to that, and haven’t. But if they’ve discovered some way to just convert matter directly to energy, stellar energy might be passe for them, and they could be flying around their stars using relatively invisible energy sources.