The villager profession are unable to be removed by removing a workstation, they keep the profession permanently. I tried to get a specific librarian I want, I gave it a workstation, then removed it and the villager is still a librarian. I did not trade with the villager.
I am not having an issue with this on 20w10a. Villagers will only release their occupation during mid day. The sun has to have gotten a bit into the sky. Villagers will hold their occupation until the next morning regardless of their job site block once the gathering hour has begun which is jut a bit before the sunset begins. Once it is morning again, the villager reaches job site searching period, realizes their job site block is gone and they become normal villager again. You may need to place and destroy their job site block to wake them up. Silly villagers hahahaha!!! You can check the villagers schedule on the wikis