How to filter out overlapping audio? Given a mixed track and one stem track, what's the best way to "subtract" the stem from the mix?

Question

Given two audio streams, I'd like to remove the audio in one stream that also exists in another.  In this particular case the 1st stream contains everything mixed, and the 2nd stream contains just the background noise which also exists in the 1st stream.  Is there a reasonable way to go about "subtracting" this background audio track from the 1st audio track?
Ideally using ffmpeg, but open to anything.  I've naively tried inverting the background channel and mixing it with the first channel with this ffmpeg filter: aeval=val(0)|-val(1) but that doesn't work.  I assume that's because the background is mixed at a different level. (I'm also assuming they're exactly in phase)
Any other ideas?

For reference, here's ffmpeg showcqt filter on each channel side by side.  You can see how the 2nd column seems to "contain" all of the audio in the first column, plus some more.  (and mixed at different levels)

Peter Barton · Answer

Inverting the background should work, that's exactly how a balanced audio connection works to eliminate noise, but how exact a match is your isolated background track compared to the mixed track? It's only going to fully work if it is recorded at the exact same quality, volume, etc, and the timing is precisely the same. What was the recording source for each track? How confident are you that the two sets of background audio that were recorded are identical to each other?

Answered by Peter Barton on October 28, 2021

How to filter out overlapping audio? Given a mixed track and one stem track, what's the best way to "subtract" the stem from the mix?

One Answer

Add your own answers!

Ask a Question