Raids not starting when you drink a ominous bottle

I had already built a raid farm and had a ominous bottle ready to test out the farm. When I drank it, it made the usual sound but the raid didn't start. I thought it was some glitch and restarted the game, but still nothing happened. When I drink an ominous bottle far away from the raid farm, the normal stuff happens, I get bad omen, but as soon as I get to the raid farm the effect disappears and nothing happens. I've already used 5 bottles.