Dirt blocks don't immediatly break when I break them

Recently, I have been trying to make an egg/chicken farm, where any eggs are laid are dispensed back into the pen, hopefully spawning a baby chicken 🙂. I put down a lever to switch on two powered rails, but I now use this if my idiotic horse lumbers on the track and stops the mine cart. To get to it, I have to break a dirt block above the lever, About half the time, the dirt block breaks, but doesn't drop a dirt block and becomes a solid indestructible black block for about 2 seconds. Can you fix this - it is getting really annoying and I don't want to kill my horse