weird redstone powering it self

while messing around with Redstone I put a repeater into a block and put Redstone connecting to it from the back then to the side then to more towards the block and powered it, but when I unpowered it, it stayed on powering its self.