Item frame doesn't drop item if you click it

I placed a item frame but I wanted something else inside and I couldn't get the item out of the item frame anymore..

What I expected to happen was...:
Drop the item out of the Item frame

What actually happened was...:
It does nothing, It didn't pop out of it, I clicked it with left mouse a lot but the item inside the item frame doesn't come out(You can break the block and then it can go away)

Steps to Reproduce:
1. Put a item frame on a block/floor/ceiling
2. Put a item, any item (I did with Pigstep music disc and grass block)
3. Get the item out of the item frame with clicking.