Wonky credit score situation, when should I refinance?

Back Story

I had 750+ credit in February when I decided to look for a home.

• Searched for a home w help from a friend and my brother
• Worked with the agent to set everything up
• The sellers were looking to retire and asked for additional "non refundable" earnest money to ensure I was a serious buyer ( I didn’t see an issue with this )
• Worked with the Loan Officer to get all my financials in order
• My credit came back at 640 – it had dropped 120 points the day before

It turns out, due to the error of another bank, it had been misreported that I’d had a "missed payment" on another asset for may. Said bank would not release in writing that the issue was on their end which made me ineligible for a rapid-rescore so I had to wait for their internal review which would have caused my earnest money to expire thus losing it all.

SO

I put an additional 10% down which brought me down to a 3.6% interest rate (not bad). About two weeks after the new home was signed & sealed the other bank’s internal review was done early and the credit bureaus were showing me at 750+ again, unfortunately this didn’t help me at all.

I talked to the original loan officer and we decided I have to wait short of a year before I can do a refinance and we aren’t sure that’s going to be helpful anyway. The biggest loss is that I’m paying an additional $100 a month for this home.

Question

What can be done in a situation like this? Is there anything I can do to negate that additional $100/mo or am I simply out of luck ?