How to extinguish yourself in the Nether?

You don't, you just burn to death with serene acceptance. Or flailing, that works too.

On lower difficulties (normal and below, I believe) a fire can go out before it kills you, but you need to be starting with pretty close to full health, which is hard when you were just hit by a fireball.

Ways to survive (or at least have the important things survive):

• Don't take valuables with you until you've got potion-brewing set up. At least then when you die you don't lose much and you can quickly get back there with fresh (crappy) equipment.

• Wear armour enchanted with fire protection and cross your fingers that it'll prevent enough damage to survive until the fire goes out.

• Use cheats to switch to Creative mode.

• Install mods that add items that will help and be craftable before you get potions. (Death Chest in particular would help, but so would healing wands or whatever million different mods are out there.)

That's pretty much it. "Accept and plan for death", "wear magic armour", and "cheat" is the sum of how to deal with burning to death in the Nether without potions.

Use potions. You can use a Potion of Fire Resistance to negate lava/fire damage, or maybe a Potion of Regeneration/Potion of Health to hold off death until you are safe. Golden Apples can do both.

You can try to get somewhere safe and die there/put items in a chest so you can come back after you respawn and get the items.

If you don't mind cheating, type "/gamemode 1" to enter creative mode; flying, invincibility, and infinite items can be achieved with this. If it says "You don't have permission" or something, and your on singleplayer, press "Esc" (Escape), Click Open to LAN, Enable "Allow Cheats", and Click "Start LAN world." Now type /gamemode 1 and it should work.

After the Minecraft Wiki:

• If you're on fire and travel through a portal the fire will be extinguished.
• If you stand in an unactivated Nether Portal and light it, you will be instantly teleported (barring load times).

That's not a very portable way, but building a portal near a spawning spot of blazes could be helpful in obtaining the first blaze rod and provide a safe escape route - and then (once you found some nether wart too) you can get to creating potions; while fire resistance will depend on finding and killing magma cubes, you can move around the fortress unmolested with use of a potion of invisibility. (which, outside of nether wart, requires only Overworld ingredients.)

Also, outside of potions, golden apples improve regeneration too, which might help your chance to survive.

You can dump water at your feet, Though the water will instantly vaporise to nothing, the split second of existence after it is placed can be used to extinguish oneself.

Or you can use fire resist potions and armour to not burn as much as without.

EDIT: This answer is outdated.

Get an enchanted apple. That will give you fire resistance among other things. That is the way to be safe in the nether without having to go there first to get the necessary potion ingredients.

Always bring snow golems (snowmen) to to the nether. They may not stop the blaze's fire, but they will take it's attention away from you.
To make one, place 2 snow blocks vertically and a pumpkin on top.

I don't know if this works. This came off the top of my head when I saw the question.

Water can be used in the nether. Kind of.

Placing a cauldron down, then placing a water bucket in it will allow water to be used to put oneself out, as stepping in the cauldron will remove a bit of water, putting you out.