Why is 'lavender' used to reference homosexuality?

Lavender, an adjective used to refer to homosexuals is from late 19th century according GDoS

(orig. US) a euph. for homosexual and anything referring to homosexuality; also as n., homosexuality.

• [1870 [UK] ‘The Ninety-Ninth Hussars’ in Songs for the Army 46: Sir Lavender Silk was a pretty young man, [...] / His men, though respectful, had thoughts of their own / Which might have spoke out if they chose, / That Sir Lavender Silk had the aspect alone / Of a Lady dressed up in men’s clothes!].

but usage examples become more common from the ‘20s

1928 [US] M. West Pleasure Man (1997) Act I: stanley: And don’t you annoy the boys, Violet. paradise: Lavender, maybe, but violet never.

According to the site out.com the lavender color was used in early 20th by lesbian women’s movement:

Bill Leap, perhaps the world’s most respected scholar in the field known as lavender linguistics, talks in a Southern drawl and cusses like a trucker’s wife.

In 1993, Leap created the Lavender Languages and Linguistics Conference, now in its 24th season. The two-day event draws about 150 attendees from all over the world and is the longest-running LGBT-studies conference in the U.S., and the only one dedicated to language issues, according to Leap. In 1993, much like today, the community squabbled over language politics, starting with what to call the field of study — queer language? Gay and lesbian language? Leap went with lavender.

“I thought, Let’s use that ancient term ‘lavender’ and let’s offend everybody,” he says. Lavender, he points out, has been associated with the occult and mysticism, with women’s power in Africa, and with forms of power in the West in the Roman Imperial Court and the Catholic Church.

“It surfaces in the 20th century with a lesbian women’s movement in England, which was marked in public by women who wore lavender-colored rhinoceros pins on their lapel,” he says.