I'm wondering if this could be related to the 1.5 animated textures feature, could it be updating every icon on the texture sheet instead of just those that have changed ?
Easy fix, been using this one for a while, as have others.
in EntityVillager
find:
ChunkCoordinates var1 = this.villageObj.getCenter();
Villagers cluster around center of village
replace with:
VillageDoorInfo var1 = this.villageObj.findNearestDoorUnrestricted(MathHelper.floor_double(this.posX), MathHelper.floor_double(this.posY), MathHelper.floor_double(this.posZ));
Villagers go to last door they encountered.
Kane Hart extremely disappointing that Mojang can improve support for custom textures, but then wash their hands of bugs that cripple the ability to make use of it. I too have found Grum to be useless at dealing with CPU usage/frame rate issues in Minecraft.
Forge has enough pull requests to sort through, without having a huge one to fix yet another Mojang bug. Previous attempts to put Optifine into Forge have been blocked anyway, and if the changes to fix this issue are extensive they could end up blocked as well.