Endermen stop attacking if they're attacked while attacking from the stare

After getting an enderman's attention using the stare, they come at me like usual, only when I go to engage them, they go back to being passive the second I hit them, and they don't attack me at all unless I either go back to looking at them (which just repeats the problem) or attack them while they're passive, after which they attack normally (they don't stop if I hit them)