Can ffmpeg be used to automatically tighten video by removing duplicate frames and quiet audio simultaneously?

I am working with gameplay footage from simulation games. Because they are simulations, even when well-optimized there are often framerate drops when the simulation becomes too computationally intensive for the hardware. I would like to remove these duplicate frames so the final video actually plays faster during these times, and at the same time remove the quietest audio adjacent to the removed duplicate frames so the audio does not go far out of sync. I have been able to do this in the past, but only by exporting the entire video as a series of images and the audio as a .WAV file and writing a program that compares frames and audio volumes and removes the parts I want removed and then reassembling what’s left into a video. However, this took an inordinate amount of hard drive space. I notice however there is a filter called mpdecimate in ffmpeg, and I’ve been experimenting with it, but I can’t quite get it to do what I want.

I’ve successfully used the following command to have it report which frames it believes are duplicates:

ffmpeg -i %1 -vf mpdecimate -loglevel debug -f null – 2> dupes

I then run a program I wrote which uses the list of duplicate frames to produce a shortened audio file with the correct length for a video with all those duplicated frames removed, but playing at the original framerate. If I export the frames as image files and then use those image files and the audio together to make a video, it works. But I can’t for the life of me figure out how to get ffmpeg to create a similar video without the intermediate step of all those image files. If I use setpts to set the framerate to what I want the new video to play at, or try to use -map to lay in my new audio, mpdecimate fails to drop any of the frames it said it was going to drop and produces a video the full length of the original instead, which of course my shortened audio does not sync with.