Word for "using other's thing without their permission"

It’s challenging to find a good adverb here, since we need to start with an adjective that means permitted, negate it, and then turn it into an adverb. The concept of permission typically requires that the permitter be stronger than the permittee, even if this is simply by appealing to social norms, e.g. it would be impolite to use someone’s phone without asking them for permission. So the adjective has to convey something about the social framework in which permission is granted, since the listener is expected (through the use of should) to respect its rules. That’s asking a single word to do a lot.

That said, there are many possible adverbs, although IMHO they don’t really improve on the OP’s original construction, such as:

• unpermittedly, or just unpermitted,
• illegally,
• uninvited,
• discourteously,
• selfishly.

Finally, is an adverb even needed? If the social framework for permission is the relationship between the speaker and the listener, what’s wrong with Don’t touch my phone.