Blocks not breaking on server (Vanilla)

At different points in the game, the game stops breaking blocks. This has happened when trying to break multiple types of blocks (ie stone, sand, logs).

I experienced this issue along with the 2 other players on my server.

The only workaround is to exit the server and log back on.