Origin of the negative connotation of "boy"

"garçon" is a neutral word nowadays. But, if we look at the history of this word, we observe that it had a connotation pejorative. If we go back even more, this word was not connoted. But, "garce", the feminine word of "garçon" is still pejorative.

Source: https://www.lexilogos.com/document/littre.php?q=Garce

On the whole, it seems that words referring to child(ren) are regularly associated with a connotation pejorative or had a connotation pejorative. Another example is the word "gosse" that is neutral in France, but in Quebec means testicle. The words "mioche" and "bambin" have also a derogative sense.

In Riffian, some argue (kossmann 2013) that the word "ahram", meaning "boy", is borrowed to the arabic word "haram" (= sin).

an elderly man might get away with calling a much younger man "my boy" when e.g. discussing something serious, as it suggests he is being fatherly

~ "come on now, my boy; you know better than that" ~

although, even then the younger man might take offence (if, for example, he believes the older man is using age to 'win' an arguement)...

I imagine adults have always been offended by those who treat them as children (/ as unworldly / as ignorant / as stupid) but simply didn't always have the right to speak up for themselves

~ in a small community where elders are revered by all, they would have been condemned for disagreeing with an older person, or with anyone deemed to have "status"... human rights have helped reduce such inequalities.

so, the only things you might 'gain' from calling someone "boy" these days are a scowl, a cuss, or a punch in the face...because chances are the person you're insulting knows that's what you're doing & has the power to retaliate. Provoke at your peril!

For example:

If you were a parent and were to say "come here boy" to your kid instead of "come here Joey" (if Joey is the boy's name) and didn't call your child by their name, it would be referring to them in the third person instead of their name, it would be like saying to your mum, "hey woman come here" or to your dad, "hey man come here", or even for example your dog has the name "kyle" and you call out "hey dog", it won't pay attention to you because it would believe you are talking to or about someone else named "dog".

The answer is, in society it would be socially "strange" to call someone boy if they knew them personally, but if you don't know the "boy" or "girl" or "person" you can call them as such.

Early etymology of boy: Wikipedia has:

Etymology:

The word "boy" comes from Middle English boi, boye ("boy, servant"), related to other Germanic words for boy, namely East Frisian boi ("boy, young man") and West Frisian boai ("boy").

Although the exact etymology is obscure, the English and Frisian forms probably derive from an earlier Anglo-Frisian *bō-ja ("little brother"), a diminutive of the Germanic root *bō- ("brother, male relation"), from Proto-Indo-European *bhā-, *bhāt- ("father, brother").

The root is also found in Norwegian dialectal boa ("brother"), and, through a reduplicated variant *bō-bō-, in Old Norse bófi, Dutch boef "(criminal) knave, rogue", German Bube ("knave, rogue, boy"). Furthermore, the word may be related to Bōia, an Anglo-Saxon personal name.

and The Online Etymology Dictionary [re-ordered]

[boy] mid-13c., boie "servant, commoner, knave" (generally young and male); c. 1300, "rascal, ruffian, knave; urchin," mid-14c. as "male child before puberty" (possibly an extended sense from the "urchin" one). A word of unknown origin.

Possibly from Old French embuie "one fettered," from Vulgar Latin *imboiare, from Latin boia "leg iron, yoke, leather collar," from Greek boeiai dorai "ox hides." (Words for "boy" double as "servant, attendant" across the Indo-European map -- compare Italian ragazzo, French garçon, Greek pais, Middle English knave, Old Church Slavonic otroku -- and often it is difficult to say which meaning came first.)

But it also appears to be identical with East Frisian boi "young gentleman," and perhaps with Dutch boef "knave," from Middle Dutch boeve, perhaps from Middle Low German buobe. This suggests a gradational relationship to babe.

In Old English, only the proper name Boia has been recorded. ME boi meant 'churl, servant' and (rarely) 'devil.' In texts, the meaning 'male child' does not antedate 1400. ModE boy looks like a semantic blend of an onomatopoeic word for an evil spirit (*boi) and a baby word for 'brother' (*bo). [Liberman]

For a different conjecture: Used slightingly of young men in Middle English, also in familiar or contemptuous use of criminal toughs or men in the armed services. In some local uses "a man," without reference to age (OED lists "in Cornwall, in Ireland, in the far West of the U.S."). Meaning "male negro slave or Asian personal servant of any age" attested from c. 1600. Extended form boyo is attested from 1870. Emphatic exclamation oh, boy is attested by 1917.)

So, from times not too long after its earliest use in English, 'boy' seems to have carried the unmarked sense (male child / male relation / little brother), as well as the pejorative (urchin / rascal / knave), and even approbatory (young gentleman) associations.

The problems associated with using a term that can both be unmarked, mundane and highly proper, and offensive have been found especially in the United States and South Africa [the same Wikipedia article]:

Race:

Historically, in the United States and South Africa, "boy" was not only a "neutral" term for domestics but also a disparaging term towards men of color; the term implied a subservient status.

The article goes on to describe diachronic changes in the perceived degree of offensiveness of the word:

The use of the term "boy" to describe men of color has not always been used as an insult, however; for example, Thomas Branch, an early African-American Seventh-day Adventist missionary to Nyassaland (Malawi) [1900s] referred to the native students as boys:

There is one way by which we judge many of our present boys to be quite different from some of those who were here long ago: those that are married have their wives here with them, and build their own houses, and all are busy making their gardens. I have told all the boys that if they wished to stay here and learn, those that had wives must bring them. This is having a good effect on them. They stay longer, and are more attentive to their work and their studies.

... [2010s] Multiple politicians – including New Jersey Governor Chris Christie and former Kentucky Congressman Geoff Davis – have been criticized publicly for referring to a black man as "boy.

During an event promoting the 2017 boxing bout between Floyd Mayweather Jr. and Conor McGregor, the latter told the former to "dance for me, boy." The remarks led several boxers – including Mayweather and Andre Ward – as well as multiple commentators to accuse McGregor of racism.

This is obviously a complex issue. A Welshman would refer to a mate as 'boyo' in just the same spirit as others would refer to mates as 'lover', 'me old mucker', 'old boy', 'my dear old thing'... and exception could doubtless be taken to all of these. 'Pal' said in a certain way is just as nasty as others. As always, the answer to misuse is not disuse but proper use. And as always, if one does not agree on a final arbiter, deciding on which is which merely rephrases the problem.

I noticed Spanish was not discussed that much in all the comments above. You need to understand that boy is a noun, and the word chico may be used as a noun but comes from an adjective, meaning small in size. If you study Spanish you realize that diminutives -ito, -ita, and anything that is generally talking about smaller size always has positive connotations when used. French is not that different with its use of petit, petite. In fact "Chico" and "Chica" would be like saying "Petit" or "Petite" which French speakers also use. Adjectives and descriptives in Spanish are commonly used in a positive way and not disparaging. Take the words "Gordo" & "Gorda", which mean fat. They are often used as a very sweet endearment or used a nickmane for people. So an adjective such a "Chico" can end up being the nickname of a young person, either because the person is small or because the person is young. However, chico & chica somehow crossed the threshold of being just an endearment or a nickname to becoming a common word to mean people who are not old (teens & young adults). But in essessence, because it comes from an adjective, it holds that positive endearing quality.

The actual noun for boy is Spanish is niño, but it is not used for adult males ever, even in passing, unless you are insulting an adult male for not being mature. Because it is just a noun, it doesn't carry that feeling of endearment at all that chico or chica has. And again, going to French, if you call someone petit or petite it would also be very endearing, and that is something that they both have in common.

So I conclude, using boy or garçon or niño for adult males will not go over very well. However, chico and chica are different because it comes from an adjective and it does not mean boy at all.