I think it has not to do with golem spawn, because golens are spawning here, and villagers keep changing in despite of that.
Same here. I got a villager hall full of librarians, all locked because I've already traded with them. Besides that, when I breed new villagers and they grow, they sometimes attach to workstations that other villagers were attached to in first place.
Also, stephanie, vote for the issue 😉