What is the name of the phoneme produced in an upper-class Briton's pronunciation of the word "Duke"? What's different in the articulation?

I think it's simply a d with a y-glide, which can make it sound like dj when spoken quickly: Dyook rather than dook.

This is called palatalization of the /d/ sound: under influence of a /u/ sound ("oo" as in moon), the preceding consonant sometimes gets the palatal glide /ʲ/, pronounced like "y" in yoke. This results in /dʲuːk/, also rendered /djuːk/.

I believe this palatalization can occur with any /u/ sound, as in news, /nʲuːz/, but it is much less common with short /ʊ/ as in book or put, that is, I can't think of any example. It generally depends on the word and the dialect whether the consonant is always, sometimes, or never palatalized. There appears to be less palatalization in America and in Estuary English (middle-class South-East England) than in Received Pronunciation. This pronunciation /duːk/ is sometimes called yod dropping, from the name of the semi-vowel y in linguistics. There may be rules behind this, but I don't have the full picture, especially not on the elusive pronunciation of lu-.

I'm a speaker of Australian English (AusEng has the "y" sound in Duke that AmEng lacks) and an armchair linguist (read extensively but no formal study).

Phonology is a complex field with many approaches, analyses, and competing theories and though I know IPA I'm not an expert at phonology.

Having said that, in British and Australian dictionaries the sound in "duke" is simply regarded as /j/, the very same sound made by the letter "y" in regularly spelled words. In IPA it is called a Palatal approximant. As such it is distinct from the processes known as palatization and iotation (which in turn are not the same as each other). This means the sound in "duke" is not the same as the Hungarian "gy".

Now the complexity of phonology comes into play because sound systems are subject to many changes so the actual realization of such words in some varieties of English may be affected by these processes after all, and I don't know enough about RP specifically.

I think we need the help of a real linguist or at least some good texts which address this specific point.

Robusto is right. It's simply a d with a y-glide because the u is pronounced as a u and combined with the sound of the letter before it (d). It’s the same with multiple, other words. Like music or enthuse.

It’s a nuance in the British way of speaking English that seems to be lost on certain people. That's why it is sometimes mispronounced and the u gets merged with the previous letter to form a different sound. The example given in the question, the word duke, will end up sounding like djuke when mispronounced by people who fail to appreciate the nuance of the u.

When pronounced properly, duke will not sound like djuke. As Robusto wrote, it should sound like dyook.

This is because the letter u should be pronounced as a u; rhyming with the word you. When pronounced, the letter before it should be combined with the u but, not merged with it. In this example, pronounce the d sound and then the u, without a pause. It's like the words do and you. When in the same sentence, they would be pronounced separately. Now imagine that there is no gap between them so that they are written doyou. Now imagine that the o after the d were removed. That would be written as dyou. Then add the k sound on the end. Therefore, dyook. That's the way to pronounce duke correctly.

You can apply the same method when pronouncing other words, that are like that. For example, enthuse. Consider the th sound and then the u.

What’s strange is why this has generally been changed in American English to an oo sound. I say generally because this does not seem to be the case with the word music in American English. I haven't noticed any Americans say moosic.

Another thing worth mentioning is that this is not particular “Received Pronunciation”. It is common across the UK, eve for people with different regional accents.

I'd point out that, in England in particular, there is a continuum between the 'precise' pronunciation, 'dyook' and the perhaps more common 'juke' (as in juke-box). In Britain, I don't think the American 'dook' is common - maybe some people in East Anglia say it that way (?)

I would say, for the most part, it is older members of the aristocracy that tend to use 'dyook' in its purest form. I think the gradient between 'dyook' and 'juke' may have some relation with perceived social class.

^{I assume for the purposes of the question that the Original Poster's J in "djuke" has a similar sound to the J in "jump".}

1. What would a linguist call this phoneme?

The phoneme is a voiced postalveolar affricate, /dʒ/.

2. What is the difference in articulation between it and a "regular" /d/?

This sound has an onset similar to a /d/ but retracted to the edge of the alveolar ridge. This involves a complete blockage of the air leaving the vocal tract. The air builds up behind the blockage increasing the air pressure within the aural cavity. Whereas with a /d/ this air would be released rapidly causing sudden audible plosion, in the case of /dʒ/, the air is forced across the surface of the tongue and through a narrow aperture between the tongue and the alveolar ridge. Along with the continuing vocal fold vibration from the larynx, this causes voiced audible friction.

Further information

Occasionally the adjustments we make to pass smoothly from one sound to another result in a new sound replacing the original two. This is known as coalescent assimilation. In both Southern Standard British English and also in General American this can happen when we have a /d/ followed by a /j/. The resulting sound is recognised as /dʒ/, the first sound in the word jump. For example in the sequence would you, we are likely to hear wouldju /wʊdʒu/.

Southern Standard British English often has an intervening /j/ between alveolar consonants and a following /u:/ in words where most General American speakers use only the initial consonant and the /u:/. For example the word news is /nju:z/ in SSBE, but /nu:z/ in general American.

For this reason many words that begin /du:/ in General American begin with a /dʒu/ in SSBE for many speakers. This has nothing to do with class.