What are natural ways to express 'contra-positively' in writing?

Conversely would be perfectly acceptable here, as the word in common parlance simply suggests some type of reversal or contrast, rather than the very specific meaning it takes in the domain of mathematical logic. There doesn't even need to be any kind of if-then statement to use the term:

Online sales went up last quarter, while in-store sales, conversely, went down.

As long as you have two statements that stand in stark contrast or represent some kind of reversal, the use of conversely is generally appropriate, even if it is not a true example of a logical converse. That said, if you are writing a thesis that deals extensively with logical converses, it may be better to use a different term.

If one is writing for an audience that knows the meaning of contrapositive(ly) it is, of course, best to use that term; no other term can convey quite the same idea. If one is writing for an audience that is unfamiliar with the term, one simply has to sacrifice some of the meaning that would be conveyed by it. Now, if two propositions are the contrapositives of each other, then they (1) look very different from each other, but are nevertheless (2) logically equivalent. The solutions proposed in the other answers emphasise (1), and sacrifice (2). If something needs to be sacrificed, it would be, however, better to sacrifice (1), which is obvious anyway, and choose an expression that, at least partially, conveys (2). Some of the expressions that may accomplish that are therefore and in other words.

One option is on the contrary, an idiomatic phrase that can be used to introduce the opposite idea (Merriam-Webster, under contrary):

As is often the case, one can get so involved in complex formalism, that one forgets the bigger picture. On the contrary, being too informal leads to interpretation and ambiguity.

Here is an example I found in a quick search on JSTOR, from Moquet et al., 2021:

The host range of specialist species that only share a few host species with B. dorsalis did not change significantly. On the contrary, we observed a significant shift in diversity or proportion of host range and climatic niches for the generalist species, such as Bactrocera zonata, Ceratitis Quilicii, and Ceratitis capitata.

The single-word version is contrariwise, which sounds more old-fashioned but literally means "on the contrary" (Merriam-Webster).

Similarly

From Wikipedia:

If a statement is true, then its contrapositive is true (and vice versa). If a statement is false, then its contrapositive is false (and vice versa).

So what you want is to express a positive correlation: If this is a square, then it has 4 sides. Similarly, if it doesn't have 4 sides, then it's certainly not a square.

Other options

• In the same vein
• It follows...
• Therefore...
• (We see/One sees/Clearly) then...

And so forth.

If the logical fact that two statements are contrapositives is important to emphasize, then I would call more attention to it. It is likely to be somewhat stilted, but perhaps that is worth it for the sake of making your argument logic explicit. It also makes an assumption that your audience understands the distinction and will benefit from having it pointed out. Here is an example:

All real men love watching curling. Note the contrapositive: if someone doesn't watch curling, it can be concluded that they are not real men.

If you can sacrifice some precision but want to preserve the sense that the second statement is just as true for the purpose of argument, then a more natural but more vague wording might be okay. Particularly if your reader doesn't remember the definition of contrapositive.

All ducks swim, so it follows that animals that don't swim are not ducks.

Just prefix the contrapositive phrase with "In other words, ". Avoid the word "contrapositive" and its cognates unless you're a logician.

"On the other hand" is a perfectly good solution, but "conversely" is fine as well. It is simply not true that "conversely" in English means that what follows is the logical converse of what precedes. The noun "converse" is an uncommon term that is generally used in its technical sense. The adverb "conversely" is quite common and means, to quote Merriam-Webster, "in a contrasting or opposite way." It is best not to think of it as the adverb form of the noun "converse" - that will lead you to think people are misusing it when they are not.