Cant connect to friends on Minecraft windows 10 15.0 through xbox

I am trying to play on my friend’s world on Xbox from Minecraft Windows 10 edition 15.0 and I get stuck on locating server, then it says can’t connect. But my friends can join me. I don’t want to be the host all the time when we play. I have the latest version of windows 10 and I keep all my apps updated. I don’t know why this is happening and I wanna know of any fixes.