What do you do with dielectric grease on spark plugs?

tl dr: You've hurt nothing by putting more than a little bit of grease in the boot ... don't worry about it.

Dielectric grease serves two purposes:

• Prevent electrical leakage
• Keep the boot from sticking to the porcelain of the spark plug

According to Wikipedia:

Dielectric grease is electrically insulating and does not break down when high voltage is applied. It is often applied to electrical connectors, particularly those containing rubber gaskets, as a means of lubricating and sealing rubber portions of the connector without arcing.

It's specific design is to help ensure arcing does not occur. It also helps keep water/moisture out of the boot as well, which can cause electrical leakage and ultimately arcing. Remember that direct current (what we find in cars) will find the easiest route to ground. Ignition voltage ranging around 40k-50k vdc can be considered high voltage. Putting the extra grease in the boot has only made it so you shouldn't have either of the issues which dielectric grease is meant to prevent. If it squishes out from the boot as you push it on, you've probably used too much. A little dab may only prevent the boot from sticking, but will it actually prevent electrical leakage? Probably not.