Villagers despawn after being traded with for hundreds of days

I spent a few days in the nether building an enderman farm and when I came back to my librarian villagers 4 of them (which were all on one of the 3 walls I have villagers at in this building) despawned.