Villager Pathfinding Issue

I found a really weird pathfinding bug that only affects some of my villagers (i have also included a video of what's happening). I recently got an artificial village setup and the way it works is there is one main building where all the villagers sleep. Their workstations are located outside of the building. 

During the day everything is fine, everyone goes to work and the unemployed hang around, that sort of thing. When nighttime comes around most villagers go inside and go to sleep just fine except for a few stragglers. As far as I'm aware the stragglers are different and there are different amounts of them every night. Some stragglers just take a REALLY long time to find their beds and others I have to cut a hole in the wall and push in. 

Its as if the stragglers are trying to go to their beds though the wall instead of through the door.  There are no gaps at all in the structure, the only way in or out is the doors. Any help or suggestions would be greatly appreciated as its a massive inconvenience to have villagers die every night. Here is the video: https://youtu.be/wTQfihzKOKQ