Unable to get into sewer in Stardew Valley Multiplayer even with the key

So there is a bug where only me as the host can go into the sewer as well as the desert mine even though it shows both of us having keys in our wallets, but my friend is not able to. Does anyone know why that is?