Player moves while opening inventory

I have had this happened many times! I will open my inventory and and my player while still in my inventory will move like 10 blocks. I kid you not I almost fell from a project I was doing in the air. For the record my friend has also experienced this. Not sure if this is a my world problem or game.