What's "less" in nevertheless and nonetheless?

OED

Less.

II. Noun uses, as a comparative corresponding to little pron. and n. III.

3.a. With the. That which is smaller or less important (of two things compared). Now somewhat rare.

1956 E. S. Duckett Alfred the Great ix. 175 All, therefore, writes Alfred, is of God, the greater and the less: the sun and the little stars..; the tiny brook which runs to meet the river and the river winding toward the sea.

Non/none – a negative

Nonetheless – not being smaller /lesser / of lesser importance.

This may be simplistic, but surely nonetheless, nevertheless, notwithstanding (all adverbial conjunctions, I think) imply that the statement that is to follow (that which is about to be conjoined) is not any less true or of lesser value than the statement that precedes it (that which it is conjoined to by the conjunction). re Question #2, I don't think it's possible to break a word apart and track the etymology or function of that part in isolation. That leaves your Question 1, so far, unanswered.

I don't have a complete answer, but a notable tangent in German desto as in nichts desto weniger "nonetheless" and nichtsdestotrotz "despite".

Ger. desto is cognate with Engl. the, specificly Old English þæsþe, from genitive+instrumentalis. It's used as conjunction in comparisons.

I find notable overall how particles like articles are used in comparisons, e.g. in the superlative "the biggest", likewise in French "le plus grand", also "the more the merrier", etc. p. p. The latter is indeed from þæsþe if I understand correctly (en. wiktionary, DWDS). Compare this with the German "Desto mehr desto besser" (which is however uncommon, rather je mehr desto besser, um so mehr...), and similar to Engl. so far so good. Note that so was in one paradigm with the, ultimately from Proto-Indo-European *so ~ *to.

The match between these phrases is not exact, none the least because none and nicht aren't directly cognate (unless through French non "no"). Whereas ne- "negating particle" + ein "an, 1" gives nein "no!" in German, we use kein "none, no", and that even in some idiomatic comparatives, e.g. kein schöner Land "no better land". Arguably the determiner already serves a similar role as the definitive article would to establish the absolute fact of comparison.

The genitive comes into play with the negation. First you have to consider that constructs like a man of small hight has no good reason to use "of" or, coralary, the genitive in German (groß-er Statur, der größten Statur). Moreover, we see that one, none (kein) etc. can be interpreted as pronouns. So posit e.g. there were none of a greater hight, in which case none is clearly a personal pronoun. In pragmatic discours, we might personify arguments, too, even if the argument is literally nothing. In this view, the phrase none the less is grammatically ambiguous, as either part could stem from a noun phrase: Where we might say She said nothing else, I would consider if She said none the less hadn't worked as well.

Another topic here is nominalization of adjectives with -er, which might be ambiguous with the comparative--not anymore in English. Meyer-Brügge (textbook: F 323) argues that adjectives came to Proto-Indo-European from "appositven Nomina", quoting Irene Balles to the effect that the underspecified difference between noun and adjective was not lexical, merely a syntactic one.

That said, to sum up your questions:

1. Rather, how exactly did never/none + the + less compound to mean what they do?

We speak of fossilization. You say you know what they mean, but since they are cinjunctiins, it's more important to know how to use them. The glosses quited from the OED certainly don't explain that. Conjunctions tend to be less than stable, so the fossilization probably happened rather late. The original was probably transparent, but became ambiguius through idiomatic usage (compare could care less). I pointed out that it must have been a form of comparison initially. Beyond that, we should look at quotes from the OED, if you can find any.

2. Particularly, is "less" here an adverb, pronoun, or determiner?

If nonetheless is concieved of as one word, then it makes hardly any sense to ask for the POS of its compounds. I tried to argue that it was ambiguous and that it is again underspecified, now. The whole phrase is a conjunction or adverb. So are its constituents, if you will.

3. What does "less" refer to? What's "less" than what?

In my above (re) construction treating it as an adjective, it refers to the aforesaid, simply the next possible referent. If none is available, it can be interpreted as rhethorical question. Out of context it cannot be interpreted. Mike is bigger - - bigger than who? In that sense, also compare unless, lest.

More interestingly, if taking the less as a noun, then none is its coreferent. Mike the great. In this sense it's almost an interjection. Jimminy Cricket, I have no idea.

Nevertheless, as I said initially, all that is just tangential information.