Pretty much what the summary says. Also, this only happens if you look at the enderman's eyes, not just simply hitting the enderman
How to reproduce:
1. Get an enderman angry by looking at him
2. Hit the enderman
3. Notice he goes back to neutral and no longer attacks you
Sometimes it takes a few hits for this to happen, but in all the 1.9 snapshots, I have never had to kill an enderman. They all go neutral after a hit or two.