Totem Doesn't Work

I was playing around with the glitch that totems aren't consumed when used. So I put a totem in my offhand, and flew up a couple of blocks in creative (Far enough to kill a player). Then, with the totem in my offhand, I fell down and I died. I had keepInventory off though when I respawned, the totem was gone. This is a vanilla server, so I doubt it was a phantom item. Here is how I did it.
1) I cleared My inventory via holding a totem on my pointer, then shift-clicking the X button on creative mode
2) I put it in my offhand
3) I flew up > 23 blocks (I had no armor on)
4) /gamemode s

I tried to recreate it and the totem worked as intended (well except for the non-consuming glitch) so I don't know if it was a phantom item.
Thank You for looking,
David.