Refinance jumbo mortgage to conforming mortgage worth it?

Here's my math. You need to start by taking the current balance and using 24 yrs 8 months as the term. At the new rate, you'd pay $244/month less. If closing costs are $5500, the break even is 22.5 months. After that, consider the return on your $29,000. Just about 10% for having paid it to the mortgage. A chunk of that 'huge savings is from you lowering your own balance. Another chunk from going back to a new 30 year amortization. The deal is good, especially if you stay long term. Just not quite as dramatic as your numbers.

Years ago, I did something similar. I had a $480K mortgage 7.625%, and was able to drop it to $380K at 5.25%. But. At the same time I dropped from the remaining 25 years down to 15. Paying down the principal so much also dropped 10 years off the back end. That was at a time when the going rate was actually lower still, and the 5.25% came with zero closing costs. To sleep better at night after depleting nearly $90K from the emergency acct, I set up a HELOC at the same time. Having a credit line sub-5% seemed wiser than having $90K earning near zero interest. Many warned this was an awful idea. And I suppose a number of things could have gone wrong. I only offer this anecdote to share my own experience.

In the end, I'd then keep the 2.875% as long as possible, i.e. don't actually pay on the 22 year math, keep it to 30. Use the savings to kill any 5%+ debt you might have, and invest the rest for retirement. The actual $8K/yr cash flow will help you replenish savings if needed.

UPDATED ANSWER -

We need to eliminate the effect of refinancing for a longer term. I have looked again at the numbers and 24.75 remaining time seems right.

You save $385.90/mo for a cash outlay of $34415.

Since this is for 24.75 years, it's treated like an annuity, i.e. the flow of money isn't forever, it's a fixed term.

To get that cash flow from $34415 would require a return of 12.89%.

Of course, that assumes you keep the mortgage for the full term. Which is why people also say $5500/$386 = 14.25 months to break even on the closing costs. In the end, I suppose one can calculate the return for shorter end times as well. Obviously, selling the house at about 14 years or sooner gives a zero return as you only gained enough to cover closing costs.

Again, I'd not suggest you actually make those payments. At sub-3%, I'd use the extra (cash flow) of $672 more wisely.