Sometimes when I join a world on Xbox then, when I'm not the host, all my inventory is deleted and I am sent back to spawn. It is really annoying because when it happens I don't have anything in my inventory I had before and I'm back at spawn so I don't know where I was before I left. The same time when my inventory disappears, another account spawned somewhere they weren’t. He was exploring when he left, and when he rejoined, he had a different inventory and was in his mine, not exploring.
Make sure that Player 1 Fully quits the game, like fun the home screen, then reloads it, and then you join and you'll have everything. 🙂