How can a magnet in a magnetic field have a potential energy, when the magnetic field is not conservative?

Even though the magnetic field is a non conservative field, the potential energy of a magnetic dipole (current-carrying log or bar magnet) is defined in a magnetic field. How can this be possible? Please explain.

From what I have learned from reading other answers, the magnetic field, in some cases, behaves like a conservative force field.
It’s called non conservative due to its field lines forming closed loops.
Its nature is not exactly defined due to it acting at right angles to the velocity of charge particle, which is different than other generally known forces, and due to the non existence of magnetic monopoles.
But this didn’t exactly answer my question.