Yup, even pillaring up causes huge fall damage. After about 30 blocks high using leaves or gravel it'll smack you for 8 hearts damage even though nothing is hitting the player, as others have said ender pearls also have killed me from phantom fall damage.
I think this is probably the same glitch as how blocks don't break when mining. Some sort of a desync.
Hey thx Jacob that helps!
Yea this drives me crazy, It looks like I don't know how to menu for my streams and it kills my speed runs. It will not swap with each other, other items lock so bad you have to throw the item on the ground with the instant discard button outside of the menu to fill it with something not on the hot bar, they cannot be switched left or right on the hot bar, completely locked.
It happens at least once every speedrun