Have had same issue playing via XBox One on Realms (Craftwerk); logged out/back in while within a hollowed pillar, may have had an effect? I have at least four at this point. Main thing was happened not only from death and re-entrance to the end.
Though it remains unclear as to whether this happens more often due to switching an item out of hand for a new inventory item, I seem to lose tools/consumables mainly... Four swords, an axe, a partial stack of pies, but also a single chiseled stone block...
I finally beat my four by tunneling inside the pillars and taking them out via bow and arrow as they'd fly into or from the center pillar, fwiw.