Animals walking through fences

Occasionally I have found some of my animals in the wrong pen. I have a building devoted to cows, chickens, pigs, and sheep. The building is divided into 4 corners with fencing dividing the sections. I have a walkway running down the middle so it is 2 pens on the left, and 2 pens on the right. On the left I have the cows and the pigs and I have occasionally come across this bug under 3 circumstances: Starting the game, returning from a mining run, and returning from an exploration run on the surface.

This issue does not happen every time, but it is frequent enough to be noticeable. It is kind of hard to miss pigs in the cow pen, cows in the pig pen and chickens in the sheep pen.

It is a roofed structure with the walls being 2 blocks tall and a roof on the third block height.