Unable to move the screen some times on the touch screen

When I play, I suddenly can't move the screen until I lift all my fingers. This is very annoying and makes me very bad at fighting and building quickly.