Iron Golem repeatedly try to jump onto fences that should be too far away

I ended up starting a save on an island in which I eventually moved a village to it and decided to fence off the island to keep Drowned from overrunning my village and house. Most of the land is one block above sea level so at first I left a 1 block space between the fence and the land (the block is at the same level as the fence) but then I realized Iron Golems can walk over 1 block holes so I went with 2 blocks between the fence and the grass.

But then I kept seeing the Iron Golems on the other side of the fences. I could bring them back inside the fence but then once I turned around they had gotten stuck again. I ended up watching one of them try to jump from a corner to the top of the fence post over and over again, like it REALLY wanted to get on the fence. And at that time the corners are 2 or 3 diagonal blocks away from a corner fence, but they keep trying to get on it and eventually succeed because nothing tells them to stop. And then the Iron Golem will try to walk on the fence and usually falls on the other side between the fence and water so it gets stuck.