Neutral Endermen have provoked facial expression

Shortly after starting a new world, I accidentally looked at an Enderman. As I only had a stone sword and no armor, I fled and the Enderman eventually teleported away. Now every Enderman I find appears provoked (has it's face in the mouth-open expression), but does not attack me or play the sound.