How can I reduce hi-fi amplifier latency?

I’ve currently got a semi-home-studio cobbled together using bits and pieces that I had lying around. I’ve noticed that when I play my instrument through speakers, there’s noticeable latency, about 40ms. When I listen through headphones directly from the mixer, there’s no noticeable latency.

The setup is like this:

Instrument(s) -> Behringer Q802USB mixer -> Pioneer XC-HM51 mini hi-fi -> Speakers

Headphones at either the instrument output or mixer output has minimal latency, but headphones at the hi-fi output (or speakers) has latency which makes me suspect the Hi-Fi. Do I need to find a different hi-fi system, or use a dedicated amplifier?

FYI, I measured the latency by having a microphone between my electronic snare drum and the speaker, and measuring the time between the sound from hitting the snare, and the sound coming out of the speaker. From one hit, there’s two peaks about 40ms apart.