Is there a requirement to delete already illegally downloaded music?

Destruction of infringing articles is an available remedy for a copyright holder under US Copyright Law. If this remedy is requested by the copyright holder and the copyright holder is the prevailing party in an infringement lawsuit, the Court may order the destruction of infringing articles.

In 17 USC § 503. Remedies for infringement: Impounding and disposition of infringing articles:

(a) At any time while an action under this title is pending, the court may order the impounding, on such terms as it may deem reasonable, of all copies or phonorecords claimed to have been made or used in violation of the copyright owner’s exclusive rights, and of all plates, molds, matrices, masters, tapes, film negatives, or other articles by means of which such copies or phonorecords may be reproduced.

(b) As part of a final judgment or decree, the court may order the destruction or other reasonable disposition of all copies or phonorecords found to have been made or used in violation of the copyright owner’s exclusive rights, and of all plates, molds, matrices, masters, tapes, film negatives, or other articles by means of which such copies or phonorecords may be reproduced.