Word meaning 'a system of unnecessary rules'

Bureaucracy

: a large group of people who are involved in running a government but who are not elected : a system of government or business that has many complicated rules and ways of doing things (m-w.com)

Micromanagement

: to try to control or manage all the small parts of (something, such as an activity) in a way that is usually not wanted or that causes problems (m-w.com)

And finally, an allusion that really sums it up: "On Wednesdays, we wear pink."

There is an idiom called Red tape that is closely related:

Red tape is an idiom that refers to excessive regulation or rigid conformity to formal rules that is considered redundant or bureaucratic and hinders or prevents action or decision-making. It is usually applied to governments, corporations, and other large organizations.

Examples and a definition from the book "Idioms in the News - 1,000 Phrases, Real Examples" By Peter Bengelsdorf:

If you are ok with American slang: this is a TPS report.

After its use in the comedic 1999 film Office Space, "TPS report" has come to connote pointless, mindless paperwork,[2] and an example of "literacy practices" in the work environment that are "meaningless exercises imposed upon employees by an inept and uncaring management" and "relentlessly mundane and enervating".[3] According to the film's writer and director Mike Judge, the acronym stood for "Test Program Set" in the movie

Not exactly what you asked for, since you seem to be specifying a noun, but "Kafkaesque" is a kind of pretentious adjective that's often used to characterize a place or experience as involving lots of alienating, oppressive, and/or otherwise pointless and circumlocutory rules (things that characterize Kafka's works, like the novel The Castle, in which a man is endlessly held back from making a simple entry into a castle by a succession of ridiculous rules and regulations).