Oskar Ahlberg

This is not a bug but a feature.
It's called quasi-connectivity. It causes dispensers, droppers and pistons to activate when the space above the block is powered and the block is updated. The block will remain active when the space above loses power until the block is updated again.

This is because Observers output a pulse that's too short for the Redstone Torch to turn off, as such this is working as intended.