Thats not a bug, You have to wait after a specific time before it can unclaim its job. I dont know the time but im pretty sure its 5 PM (In-Game Time) and also dont kill the villagers unless you have a breeder and its close to the place you want to deposit. But another reason (Which you didnt do) is if you trade with it which permannently locks it in.
Maybe your the monster, Just kidding, did you check if theres pillagers?
After recording this video, I put a command block on a timer and did a similar thing with a pre-existing recipe. I gave myself a raw rabbit, revoked the recipe so it would be granted by collecting it from the furnace, and I put it in. Once I had put it in my inventory, and the timer on the command block ran out, the recipe was removed from my recipe book just as expected. I removed the function and tried the same thing with my custom recipe and it was removed as expected. I hope this helps