Pronoun Case: When I was a child, Grandpa was unhappy with (me/my) excavating his rose garden in the hopes of finding dinosaur bones

Taking a stab: We know that whatever comes after the word "with" has to be a noun, and essentially has to be the reason why Grandpa is unhappy. If Grandpa was just disgruntled by the person writing in first person singular, then "me" would suffice, but technically it is the action of excavating, or the nominalization of the verb, and so the possessive pronoun is more correct, since me is not a possessive pronoun. I am stretching my syntax knowledge here.

To give practical examples:

She does not like my jacket. -> my jacket is a noun phrase and object of "like"

*She does not like me jacket -> obviously wrong.

She does not like singing (singing is a gerund and object of "like")= She does not like the action of singing (the object of "like") (The context will tell you if she does not like to sing, or if she does not like to hear anyone who is singing.)

She does not like {my singing} (Verbal noun phrase and object of "like") = She does not like {the sound that I make when I sing.}

She does not like me singing = She does not like me when I sing (but she might like you if you sing.) Correctly, this would have a comma after "me". (It is also possible to understand this as "when she sings, she does not like me.")

In prescriptive terms, only "my singing" is correct, but so many people use "me singing" that "singing is understood as a post-positional adjective qualifying "me" = "She does not like the singing me."

When I was a child, Grandpa was unhappy with [(me/my) excavating his rose garden in the hopes of finding dinosaur bones].

Both forms are fine. The pronoun, either "me" or "my", is the subject, and the verb phrase "excavating his rose garden in the hopes of finding dinosaur bones" is the predicate.

The choice between gentitive "my" and accusative "me" depends on style, the genitive being characteristic of fairly formal style, and favoured by older speakers.

Since prepositions typically take a noun phrase as object, traditional grammar insisted that the so-called 'gerund' was correct by virtue of it being a 'verbal noun'. But those days are gone, and most people accept accusative "me" as a less-formal alternant.