Friend get's stuck on "connecting to multiplayer game" stage until the world is recreated

I've seen a lot of connection bugs, but none of the ones I've looked at are like the one I have encountered. So I create a world on my windows laptop, everything goes good. My brother can join from his laptop on our LAN, and we encounter no connection problems. Then my friend tries to join (from his own house) and says he gets stuck on the "Connecting to multiplayer game" stage. I look at the player list and his name doesn't even show up which means something is not letting him connect. I exit the world, create a second (with the same settings and seed) and he is able to connect (after a long time due to his iPad being old). He can see my worlds (and we're signed into Xbox Live), but he just can't join them until they are recreated for some reason. I don't think he has this issue when he connects to featured servers though.