Switching game modes causes player to lose inventory

I was playing in survival and had a few diamond tools and a sword in my inventory visible in the item bar. When switching to creative mode the items were still visible in the item bar. Switching right back to survival again caused the player to lose everything in their inventory and item bar!

Please do not tell me this is "normal". The diamond armor I was wearing was the only thing that I didn't lose when switching between game modes.