What's the reason for calling cheap seats at the theatre nosebleed seats?

It's an exaggeration of the altitude of said seats. High altitude can cause nose bleeds in some people.

Merriam-Webster's Eleventh Collegiate Dictionary (2003) has this entry for nosebleed as an adjective:

nosebleed adj (1978) extremely or excessively high {seats in the nosebleed section} {nosebleed stock prices}."

The two earlier answers to the posted question make clear the physiological connection between nosebleeds and high altitude—and the hyperbolic association between (for example) inexpensive seats in the upper reaches of a sports stadium, theater, or other venue and heightened risk of incurring a nosebleed.

Since the posted question asks specifically about the origin of nosebleed used in this sense. I checked to see whether any instances of this usage occurred in print before 1978. The answer is yes: published instances go back at least to 1974.

From Betty Cuniberti, "Baseball Comes Out of Hibernation," in the San Bernardino [California] Sun-Telegram (March 30, 1974):

Baseball fans are crazy, I tell you. And they're everywhere. Even in Cleveland.

They are easily recognizable by the mustard on their shirts and the argument on the tips of their tongues.

They live in the nosebleed sections of all baseball stadiums and have radios growing out of their ears. Their pockets are full of peanut shells.

This instance of nosebleed as an adjective makes the same exaggeration-based joke that is implicit in the Eleventh Collegiate's definition—and therefore perfectly anticipates subsequent usage.

Also, from Ramon Cooklis, "Fun and Games (also Music) at Hollywood Bowl," again in the San Bernardino [California] Sun-Telegram (September 2, 1978):

It's amazing what people come to the Bowl for. From the bourgeois elegance of a candlelight-and-linen picnic dinner in the boxes to the plebian pleasures of hot dogs and beer in the bleachers, you can eat. You can come to observe the crowd or be observed. Up in the dollar seats of Nosebleed Alley, you can guzzle wine, sleep, or engage in various amorous pursuits for several hours.

Evidently, an area of inexpensive, far-from-the-action seats in the Hollywood Bowl was popularly known as "Nosebleed Alley" by 1978, suggesting that the joke about "nosebleed seats" was already widely known at that date. Such informal names are fairly common in connection with performance arenas. For example, Charter Hill, which overlooks the University of California's football stadium in Berkeley, California, has for decades be known as Tightwad Hill because people who climb it on a game day can then watch the game for free.

---

Update (November 1, 2020): A further review of the Elephind newspaper database turns up a slightly earlier (hyphenated) instance of nosebleed in the relevant sense. From "Opening Night At The Garden: Ernie In The Land Of The Giants," in the [Chestnut Hill, Massachusetts] [Boston College] Heights (October 15, 1973):

The "Old Spaghetti-Eater" from Little Rhody was the center of attraction all night as he put one move on Paul Westphal in the second quarter that left the USC grad somewhere up in the nose-bleed seats. Don "The Duck" Chaney had somewhat better luck defending DiGregorio. Chaney tried to force Ernie to the outside, hoping to bottle up the Buffalo offense, but with limited success.

Even in this very early instance, the sense of the exaggerated expression is already fully formed: the seats in question are so high up in the arena that people sitting there are susceptible to nosebleeds. In this particular instance, the notion is that the Paul Westphal (the defender) responded to a fake by Ernie DiGregorio (the player dribbling the basketball) by moving so far out of position that he wound up in the upper part of the arena seating.

And from Ryan Reese, "Are Dodger Fans Really the Best?" in the San Bernardino [California] Sun (June 14, 1974):

For $1.50 it [the centerfield bleachers section of Dodger Stadium] is not a bad place to sit. You can pay $1.50 for seats in the main grandstands, but they are located either in the far corners of the upper deck or in the upper, upper deck, nicknamed the "nose bleed" section.

These various early instances suggest that the expression "nosebleed seats" became popular in connection with seating in sports arenas and stadiums, rather than in theaters. Of course, the first confirmed instance of its use is subject to change as earlier examples come to light—and it is not impossible that an instance involving theatre seating may emerge that antedates the ones from the sporting world.

GDoS has early usages from mid-20th century:

Nose-bleed:

also nosebleeder, nosebleed seats [also used fig.; the nosebleeds that can accompany oxygen deprivation]

(US) of seating, very high up, esp. in an auditorium or sports stadium; also as n.

• 1948 Post-Standard (Syracuse, NY) 25 Nov. 25/2: George Solotaire, the ticket broker [...] was asked by a customer for seats in the second balcony. ‘’, Solotaire replied, ‘do not handle nose-bleeders’.

• 1950E. Wilson 8 Feb. [synd. col.] HGentleman Georgie Solotaire describes balcony seats in theaters as ‘nose-bleeders’.