when mobs are IN fire they do not bounce back when you hit them.
when I hit the mob it should have bounced backwards.
what happened was it just stayed there and burnt.
to reproduce you:
light a mob on fire with flint and steel
if mob is IN the fire hit it and it won't bounce back
While mobs or players are taking damage they are temporarily immune to any incoming damage for a short period of time. Since the mob stands in fire they are constantly taking fire damage. That makes them immune to incoming damage, so you can't knock them back. More information here.