Villagers teleporting through glass when they grow up

I have a villager breeder in my survival world, and it seems when the villagers grow up they teleport out of the glass box that they are in sometimes. This is a really frustrating thing, as I have an entire system connected to this breeder.  I am also on a multiplayer server, and the glass cage is attached.