Most likely being another bug with the very broken skin editor, other PC players that join your world will have the host's skin.
This does not affect console players.
You CANNOT change your skin in game to "unequip" the host's skin as a potential workaround, as you must restart the game after changing the import.
I have the same problem. It is so annoying! I don't even use my custom skin anymore until this bug gets fixed!