Villagers teleport outside of village

In my village if I leave for a little bit my villagers will somehow teleport in this lake. Some even spawn on the other side of it and get lost. Most of them find their way back to the village but I still loose a few from time to time. I blocked up the lake so it would be easier for them to get back but I still have to get my boat and drive the ones who are lost them back to the village sometimes. I’m not sure why some of them will just randomly be away from the village when they seem so desperate to get back to it when they are teleported. I know I did a bad job of describing it but I’ll get a picture of it the next time it happens.