Baby horse grows after one wheat

After breeding horses and feeding the baby a wheat, it instantly grows, but then doesn't let me ride it. Then I log out and back in and it's a baby again.