Breaking blocks in certain places causes momentary freeze

Please see this video: https://gfycat.com/healthycooperativedunlin

Aside from the already reported issue about the breaking animation following the player's cursor - every time a block is broken I'm getting a pretty serious freeze. This doesn't happen in all chunks though.

It's client-side lag, but I feel I should mention I'm playing using the 19w39a dedicated server too (unmodified) hosted on a separate machine (i.e. anything the server is doing shouldn't be affecting my game client's framerate).