Item frames have a weird hitbox

If I click on the edge of the block the item frame is on it will target the block, which makes sense. But the same happens with a filled map, which takes the whole block, not just the center