Where did the "ue" in "tongue" come from?

Old English
In Old English, a language from the West Germanic family, the standard spelling was tunge (wiktionary) and the corresponding pronunciation was /ˈtʊnɡe/ "tun-ghe" (/ʊ/ as in foot). In other words, the final "e" was not mute but clearly pronounced. For instance in present-day German, you would say "Die Zunge" /ˈtsʊŋə/ (tsung-e). In both cases the final unstressed e is actually already a reduction of an older final a still present in many other cognates.

Middle English
The spelling of Middle English is strongly influenced by that of Old French in which the un sequence is uncommon. It was thenceforth often replaced by on which led to such spellings as tonge or tounge ('ou' being the French spelling of /u/ and /ʊ/) . Which can be compared to contemporary alternative spellings longe and lounge for the word long.

Modern English
With the evolution of the pronunciation of long vowels in Modern English, these spellings became highly misleading. "Tounge" for instance could be read /taʊndʒ/ (as in lounge). As a result, such spellings as tonghe or tongue were preferred. The fact that the simpler spelling tong was not preferred (in spite of the existence of such words as long) is probably a sign that the final e was not completely muted yet.
Finally, the tongue variant probably overtook the tonghe option as a result of an analogy with such words as L. lingua or languages.

Dutch
Also note that on the other side of the English Channel, the Dutch word followed a very similar path. From Old Dutch tunga it successively evolved as Middle Dutch tunge, tonghe and present-day Dutch tong.

Reference
This is the explanation offered by the OED regarding the spelling and the etymology of the word tongue.

[OE. and ME. tunge wk. f. = OFris. tunge, OS. tunga (MLG., LG. tunge, MDu. tonghe, Du. tong), OHG. zunga, zunka (MHG., Ger. zunge), ON. tunga (Da., Norw. tunge, Sw. tunga), Goth. tuggô:—OTeut. *tungôn-, held to be cogn. with L. lingua tongue, for older *dingua (as lacrima:—dacrima: see tear n.1).    

The natural mod.Eng. repr. of OE. tunge would be tung, as in lung, rung, sung (and as the word is actually pronounced); but the ME. device of writing on for un brought in the alternative tonge with variants tounge, townge; app. the effort to show that the pronunciation was not (tundʒ(ə) led to the later tounghe, toungue, tongue, although it is true that these hardly appeared before final e was becoming mute, so that its simple omission would have been equally effective. The spelling tongue is thus neither etymological nor phonetic, and is only in a very small degree historical.]