As A Semi-Resource Pack Developer, this is annoying. If add a resource pack, and remove it, java continues to use it, meaning you can not modify it in any way. (Ex: Delete Or Replace It) This is very annoying, as I have to restart Minecraft every time I make a edit to my resource pack.
Try F3+S instead of restarting your game next time/