I am having this same issue as well. I have tried troubleshooting it but cannot make it work. One thing that might be of note in addition to what Jacob stated is that when the second person joins on the xbox, it joins under the same name as the person who started the session. I noticed this when I joined remotely from my phone to the Minecraft world. Joining from the phone did not seem to impact anything and I showed up with a separate username. But the individual who hosted the world and the next user joining on the Xbox would show up as whoever the host was with a (2) after their username (player one [example_user] player two [example_user (2)]). This is the case even if the other users trying join are logged in with separate accounts. To me, it appears that the game sees the second and third user with the same username as the host and terminates the session.
Same issue here.