When looking at a ladder, it is often the block behind that is selected.
When trying to break the ladder, the selected block will switch back and forth very fast between the block and the ladder, making it very difficult to destroy a ladder, and often the block behind breaks instead of the ladder.
Also looking through a ladder block, but without looking through the ladder hitbox, the selected block with switch very fast between the block and the ladder.
See attached screen shots for evidence of bugs in block selection
steps to reproduce
go in front of a ladder
equip an axe
put your cursor on the ladder
the block behind the ladder is selected. If it is not the case, move your cursor slightly.
try to break the ladder
the selection switch very fast between the block and the ladder while you are trying to break it.
it is almost impossible to break the ladder in this configuration
I can confirm I have seen this happen as well.