This is still happening on my realm as of Garden Awakens update, not on preview
I have EXACTLY this problem and its so frustrating. Even happens as realm owner, and I've had the issue since I can remember lol. I got around it using a VPN on Windows 10 Edition but its not the best fix, and cant be done on consoles. The only way I can reliably connect on console is to use an ethernet, so if you can use that you may be able to connect.
I've had this issue on my realm since the Creaking update as well! Was there any fix found?