なんて言わないで自己申告制, meaning of で?

ないで is the te-form of ない, but it can be a continuation marker (if it's in the middle of a sentence) or a request marker (if it's at the end of a sentence).

• 見ないで!
  Don't look!
• 見ないで答える
  To answer without looking

You seem to understand this. So the real question here is "Is there an imaginary period between 言わないで and 自己申告制"? However, no one can tell yes or no only from this short fragment of the lyrics. Even when a noun follows a te-form, it can still be part of a sentence, especially in lyrics (see this question for examples: Can te-form be used to modify a noun?). To answer a question like this, you need to access the entire lyrics and fully understand the context. So I googled for the original lyrics:

今日もココロがランアウト
現実だってシャットアウト
起きたらすぐにシットダウン
試しにファイティンポーズ
とってはみるけど
維持できないなら意味ないじゃん
なんて言わないで自己申告制
これでも頑張ってる「つもり」
やっぱりめんどくさい ムリみ〜

After reading this, the role of で still looked ambiguous to me, so I listened to this song on Spotify. My conclusion is that there is a period after 言わないで. That is, this 言わないで is a request ("don't say"), and 自己申告制 is associated with the next line. Here's my interpretation:

• 「維持できないなら意味ないじゃん」なんて言わないで。
  Don't say "If you cannot maintain, it's meaningless".
• 自己申告制(では)、これでも頑張ってる「つもり」。
  According to me, I'm doing my best.

言わないで is a negative command, "don't say."