Negative questions in (English language) listening tests

I work for a company that produces ELT materials. We write various kinds of reading and listening passages, followed by comprehension questions.

Our staff writers use the "What is NOT true" question quite often. While I find this a valid form for reading questions, I have concerns about this style for listening questions. For instance, if there’s a READING passage about the Roman Empire, and the question is, "What is something Julius Caesar did NOT do," the student can scan back through the text and eliminate the options.

But for LISTENING tests, I suspect it can be quite difficult for students to remember what they DIDN’T hear. Unfortunately, our company doesn’t do any follow-up to identify any kind of validity/reliability issues.

Are there any teaching/testing professionals out there who can give any thoughts or feedback on this issue? I suspect that we should not be using "NOT true" questions in listening tests, but I can’t find any source that backs this up.