Villagers losing job after trade

I'm working with farming villagers. I placed a composter, the villager took the job, I traded with them once to lock in that job, and according to all the information I can find, they are not supposed to be able to lose their job after trading even if their original job block (in this case, the composter) is broken. However, after breaking the composter, the villager lost their farming job. The villager was in enclosed space, unable to reach any other job blocks, yet somehow connected to the lectern in my house about 60 blocks away. There are no other lecterns in the area. Now that the villager somehow became a librarian, I opened their trading menu and it still had the experience gained from the trade I did with them as a farmer. I did not do any trades with that villager as a librarian, so it shouldn't have any experience in that job as far as I understand how the villager trading mechanic works. If I'm just wrong about how the trading mechanic works, I apologize, but I read all the information I could find online and I think there is something wrong here.