What is the problem that gets worse after you try to solve it?

See tar baby.

ODO

A difficult problem that is only aggravated by attempts to solve it.

Hydra-Headed problem. The Lernaean Hydra was a sticky problem for Heracles. It regrew two heads each time one was lopped off.

An instance of a bottomless pit (meaning 4)

an entity or problem which consumes seemingly endless resources.

or figuratively a black hole. There is a StackExchange illustration for a bottomless pit in a finite universe, based on Gabriel's horn, an interesting mathematical object:

A Gordian Knot. This suggests an intractable problem that can be solved by drastic action or innovative thinking (because Alexander the Great solved the Gordian Knot by cutting it with his sword).

Something that is occasionally used metaphorically in this sense is quicksand:

a situation that is dangerous and difficult to escape from - merriam-webster.com

I would vote for "Wicked Problem" - which is not just a qualifier, but has been used in a more precise sense, especially in Social Policy Planning, as detailed in the Wikipedia entry.

The same entry also provides a comprehensive bibliography and points to related terms.

You could use aggravate, the second definition in Merriam Webster defines aggravate as:-

to make worse, more serious, or more severe : intensify unpleasantly

So as per your example: 'This problem is aggravated every time you try to resolve it.'

Another word you could use is exasperate: Merriam Webster lists this as obsolete though it is the first word I thought of after reading your question.

It's a long expression / saying but it means exactly to find oneself in a worse predicament, or make a bad situation worse.

out of the frying pan into the fire

Fig. from a bad situation to a worse situation. (*Typically: get ~; go ~; jump ~.)
_{McGraw-Hill Dictionary of American Idioms and Phrasal Verbs}

It's a Gordian Knot, in that pulling on the rope (to try to undo it) just makes the knot tighter.

Alexander the Great gave that a new meaning (by cutting through it with his sword), but that (an untractable problem made worse by trying to solve it) is what it used to be before he showed up.

I have heard situations in which every "solution" causes more problems referred to as a quagmire:

noun
1. An area of miry or boggy ground whose surface yields under the tread; a bog.
2. A situation from which extrication is very difficult: a quagmire of financial indebtedness.
3. Anything soft or flabby.

For example, see this George Soros quote:

No external power, no terrorist organization, can defeat us. But we can defeat ourselves by getting caught in a quagmire.

Tar pit is similar to Tar baby, but is a simpler metaphor to understand. The simple and obvious meaning is that touching the problem will get you stuck in it / to it. Apparently this is an existing usage of the term.

---

Tar pit

1. a hollow in which natural tar accumulates by seepage.
2. a complicated or difficult situation or problem. "the tar pit of municipal poverty"

from google's definition

---

BTW, Tar baby is a reference to a fictional character that some people won't have heard of. If you're talking to people that probably have heard of the Tar Baby story, then it might be an even better choice.

Tar pit also avoids the racist interpretation of Tar baby that apparently exists.

This fits to a degree:

• No win scenario

Kobayashi_Maru

Although your example might have a solution which is as yet unknown.

I would try paradox

It is commonly defined as a problem that made up of two opposite things and that seems impossible but is actually true or possible.

Thus by your example when attempting to open a leaf ball, it closes. Or in the example of quick sand, the more you struggle, the more difficult it is to escape.

A conundrum is a problem that apparently has no solution, as more complications appear as aspects of it are solved.

2a: a question or problem having only a conjectural answer.
b: an intricate and difficult problem

A bureaucratic / procedurally descriptive phrase is the one step forward, two steps back idiom.  This seems fairly self-explanatory; it is defined in The Free Dictionary as

something that you say which means every time you make progress, something bad happens which causes you to be in a worse situation than you were to begin with

Apart, perhaps, from 'gordian knot' as has already been suggested, I can't think of any word or compound with an established definition that exactly fits the bill.

So, it's been mentioned already but, to me, conundrum would work best in the sentence given

This problem is a conundrum which gets [more?] complicated every time you try to solve it.

Conundrum, of course, means:

Any puzzling question or problem; an enigmatical statement. - OED

Which, on it's own, doesn't really fit the exact requirement that the OP has posted.

However, I would argue that, in current usage, it is filling the gap.

A quick check on Google News returns these on the first page:

• White House Countdown: The Ryan Conundrum

• The Afghan conundrum

• British farmers face conundrum: the hearts says ‘Leave’ the EU but the head says ‘Remain’

• Promising Gene Therapies Pose Million-Dollar Conundrum

• A political conundrum

In each of these cases the 'conundrum' is an issue or question that can be described quickly, often in a single, brief paragrapgh, but which is complicated and, often apparently unsolvable.

You could also tangentially consider a minefield:

a subject, situation, etc, beset with hidden problems
_{Collins English Dictionary - Complete & Unabridged 2012 Digital Edition}

This is generally used to describe a problem that will be a lot more complex to solve than it looks, often looking unassumingly simple.

Such a problem is snowballing or is experiencing a snowball effect

McGraw-Hill Dictionary of American Idioms and Phrasal Verbs

Figurative. [for something] to become larger or more serious by growing like a snowball being rolled. This whole problem is snowballing into a crisis very rapidly. The argument soon snowballed into a full-blown riot.