Banging your head while flying pushes you

Whenever i fly and my head directly touches a block it's very challenging to get back under a block exactly at the height where your head is being banged without it pushing you back