The bug
On the village house with two levels and a balcony, the ladder on the first floor doesn't extend all the way down to the ground, leaving one block uncovered.
I am not sure of whether or not this was intentional but I found it was much easier to climb after adding an extra ladder piece myself.
How to reproduce
Seed: -4222993603967823547
Coordinates: /tp @s -1197.73 73.00 -2772.65 166.37 6.90
I suggest reverting this fix. This was probably intentional as it prevents villagers (and zombies) from actually climbing ladders. If they do climb the ladder to the 2nd floor they can't get back down. Anytime I build a structure with a ladder I leave the bottom rung off for this reason, and I know other builders do too. Essentially it requires jumping at the foot of the ladder to get on. None of the normal creature AI has them jumping to get onto a ladder.
If villager and zombie AI have been improved enough that they can get BACK DOWN a ladder from 1 floor above then ignore my suggestion to revert, but I've never seen them do it.