A piston powered diagonally creates a BUD. If the redstone on a block diagonal to the piston has a signal strenght of one, and so a redstone on top of a block above the piston isn't powered, placing said "level one" redstone will power the piston. It will unpower it if you break it.
That's the only way to toggle the piston, any other way to turn off the level 1 redstone will cause a BUD, as if the redstone above the piston wasn't there and so acting normal (emptying the dispenser hooked up to a comparator to create the signal of the appropriate strenght, preventing the comparator to emit by powering its B input, breaking the source or any redstone piece on the line that isn't the level 1 one).
It's not directionnal, and happens in both gamemodes, and will not happen with glowstone or half slab as the block above the piston.
I think it comes from the level 1 redstone placing / breaking causing an update to the adjacent redstone piece which update the block below, because it's redstone, which in turn update the piston. Although i don't know the inner workings of redstone, so take it only as the assumption that it is (and correct me if i'm wrong, i would appreciate to know more about redstone).
(Edit) Mentionned the glowstone and half slab thing.
Related issues
Attachments
Comments

Duplicate of MC-108.
It's not this. Did you even read the post ? The block should create a BUD, but does not in the condition said. I would never have listed the issue if it was what you said, i know BUDs enough for this.
It's quite frustrating to get an issue tagged as resolved for duplicate when reading it shows it's something different.
(Edit) Sorry if i sound aggressive.
(Edit2) Or maybe it is ? Does the redstone above the piston acts as the fence gates / redstone lamps did when working ? If it is i still think it should be fixed, but admit the duplicate.
It's updating the piston because the redstone connectivity changes from "dot" to "line", see MC-1263.
Yup you're indeed right. I still don't see it as a duplicate of MC-108 though, as it is the update that is the problem here, not the BUD itself (the redstone shouldn't update the piston to go along with the recent change of the fence gates, redstone lamps and such). Also not a duplicate of MC-1263 because in it the issue is the redstone powering the piston, not just updating (it oculd update just fine without powering the piston).
I apologies for the rudeness in the previous post.
Well the whole point why it works like this is because of MC-108. When that gets fixed this inconsistency will go away anyway.