Would Galadriel have been able to defeat the Witch King?

In a literal sense, you are right, she is neither male nor a Man racially in the LoTR setting. She could have "rules-lawyered" the prophesy of Glorfindel and made a valid attempt/attack upon the Witch King had she so been moved, and had significant power. But she didn't.

On the practical side:

• Galadriel's power in the books wasn't quite like the Hollywood action genre that the Peter Jackson tapped into. In the Appendix B to RoTK, we see that Orcish forces attacked Lorien three times (last on March 22, three days before Gollum bit off the ring and fell into the Cracks of Doom), but could not defeat the Power in Lorien. She was a major reason behind that. At the end of the war (after Sauron's fall) Celeborn sallied forth from Lorien, crossing the Anduin on 28 March, and began to destroy Dol Guldur. As written, it appears that Celeborn led the operation and later met with Thranduil on April 6 (the latter arriving from the Northern part of Mirkwood/Greenwood). It is generally agreed that Galadriel was of a higher level of power than Celeborn, due to both her lineage and her Ring, but in the main hers was a different kind of power ... you could call it "A power that preserves." That said, in the Appendix B narrative (versus the time line) Galadriel laid some lumber on Dol Guldur:

  “… and when the Shadow passed, Celeborn came forth and led the host of Lórien over Anduin in many boats. They took Dol Guldur, and Galadriel threw down its walls and laid bare its pits, and the forest was cleansed.” (Appendix B, Lord of the Rings).

• Tolkien's powerful women tended to wield their power in more subtle ways than a knife in the face. (If you look at the generation that Tolkien was raised in, the female action hero was a very rare trope, and writers tend to write what they know).

  • When Gil Gilad took his armies of Elves in the last alliance, end of Second Age, to fight Sauron with Elendil and Isildur's armies, Galadriel wasn't out there on the battle field. Tolkien didn't have the action hero model in mind for his heroic women, save perhaps Eowyn of Rohan and Luthien from the Silmarillion.
  • Luthien was possessed of considerable power, but she sang Morgoth to sleep rather than using force: her songs were a powerful magic in their own right. She used her cloak to disorient Sauron, who had taken the form of a dread evil hound, which helped the great hound Huan defeat Sauron, which in turn made it possible for Luthien to free Beren from Sauron's prison. (Tale of Beren and Luthien, The Silmarillion).

• Why would Galadriel resort to such crude means (melee, physical confrontation) when such worthy warriors as Glorfindel, the Sons of Elrond, and even her husband Celeborn were of sufficient merit in close combat to take down the Witch King on their own? Elves did not fear the Nazgul the way men did.

  They {the elves} do not fear the Ringwraiths, for those who have dwelt in the Blessed Realm live at once in both worlds, and against both the Seen and the Unseen they have great power.' (FoTR, Many Meetings)

• Why would the Witch King put himself in a position to be confronted one-on-one with someone as powerful as Galdriel? He ran from Glorfindel at the battle where the prophesy originated. The Witch King worked through proxies and their ability to project fear in most of the narration Tolkien provides on the Wraiths, of whom the Witchking was the most powerful.

All said and done, the hypothetical you set up isn't how Tolkien fit those two characters into his story. There would be no cage match.

It happened, didn't it? In Return of the king, it says that the forces of Mordor tried to attack Lothlorien 3 times, but couldn't because of Galadriel. I don't know who it was exactly, but I think there where 3 Nazgul among them, including either Khamul or the witch king.

Either way, Galadriel is way more powerful and though she never actually fought in the lore (except maybe during the kin slaying), she has been stated to be as athletic as any other Eldar.

I doubt the Witch King could bear to approach Galadriel. It would be sort of akin to Dracula approaching a crucifix. Dracula may be physically more powerful than the priest holding up a cross, but he would be unable to resist the effect.

It's a crude analogy but I think the same would hold true for Galadriel. The Witch King might have more physical power (or maybe not, who knows), but he would be unable to stand, or even hold his ground against her regardless, and would flee before her.

It's like a battle of wills, and her will is essentially unbreakable, plus she wears Nenya, sometimes called the ring of adamant. She's not backing down from what is essentially a human ghost.