Occupancy and financial status affidavit

We refinanced our house a month ago and now my wife has found a different house she likes and wants to buy it and we were going to use my VA eligibility to buy the new house and once we move in sell the existing house. The only problem is we signed this affidavit for the re-fi which we currently live in and we intend on moving to the new house, fix this one up and sell it. My loan officer says the Va might require a release from this affidavit. The affidavit states a lender may not refuse unless the request is unreasonable. Thoughts on this?