When you look at a ladder from the side you won't destroy the ladder but the block. It should update similar to trapdoors. For more information see http://www.youtube.com/watch?v=cSFhVL_ES3M#t=0m36s
Linked issues
duplicates 1
Comments 2
I guess this one means the same bug: https://mojang.atlassian.net/browse/MC-491
Confirmed, could recreate.