Skip slides when going back - Beamer

I have a lot of slides with lists that use either pause or <number> after item so that they are shown at different times. Some slides have up to 9 different stages for it, so after I am done presenting and someone needs to go to a slide at the beginning it will be a hassle to go through all of them.

Is there a way so that if I am going in the reverse order in the pdf, once I reach a slide that has multiple stages it will go to the frame before and skip all the in-between moments?

I found a lot of questions/answers that would use hyperref to skip to specific sections, but I just want so that when I am going back it will skip all the unnecessary steps.

Is that way the only way to do it? I am using metropolis theme and it doesn’t have buttons at the bottom of the slide.

Thanks!