On SMP When using using a shovel to destory dirt/sand no block lag, without a shovel block lags

On a survival, mulitplayer, vanilla server I and fellow minecrafters on the server, with no lag (like chests and doors, and logging in, shows no normal signs of the usual lag experienced sometimes on minecraft)but when using no tool at all to destroy dirt (bare hands) the block does not drop and leave item to pick up until 5-20 seconds after rightfully destroyed. however when I used my diamond, eff 4/5 shovel It did drop when it should have, no lag at all. This does not happen in SSP, So it could be possibly the actual server connection. but of this I'm not sure

Update (11/11/12) Everybody else on the server has been having the issue, The blocks that are affected include obsidian, Coal and other ores, Stone, Dirt, Sand, Clay and chests. Also wood and stonebrick are affected. Some ways to avoid it are to use efficiency picks/ shovels but this can still be annoying. It definitely does not happen in SSP and other things, like opening doors, and eating is not laggy at all. Some users have speeds around 15 - 20 mb/s and still get this lag. So I am confused to how/why this is happening.