I had 20 villagers (or so) with 5 Iron Golems protecting all around my town (that is protected with 3 tall walls). I went to the Nether and when I came back there was just 2 villagers and 0 Iron Golems, also the doors were not broken and there wasn't any signal of zombies or another hostile mob. This happened more than once, and always there was just 2 "survivors" villagers.
This is exactly what happened to me.
I had 20 villagers (or so) with 5 Iron Golems protecting all around my town (that is protected with 3 tall walls). I went to the Nether and when I came back there was just 2 villagers and 0 Iron Golems, also the doors were not broken and there wasn't any signal of zombies or another hostile mob. This happened more than once, and always there was just 2 "survivors" villagers.