When the menu is not open, Minecraft is recreating textures every single frame and/or tick, which causes immense performance penalties when the textures are high res like 128x Sphax.
Edit: I rewrote this hopefully explaining this a bit more clearly.
This is a Major issue when using High Resolution Texture Packs and might even have impact on even on 16x though it's a bit hard to tell. Try a 128x or 256x for large impact testing. Pretty much go into the game with a 128x texture pack and see if your fps is cut 10 times more or even in half. At this point hit esc and see if the fps goes higher. This where it seems to be a issue with glTexImage2D.
There seems to be a major issue with Rendering on the screen when it comes to glTexImage2D. I attached a profiling of the 256 texture and it seems when not in the menu it's eating CPU time like crazy. My guess would be that it's refreshing some the textures or maybe all of them ever frame. When you escape and go into the Menu this seems to solve the issue.
I do realize that when you are in the menu a few the ticks and such do shutdown. But the texture pack is clearly still there and loaded as well the chunks are still loading. Yet you go from 1 fps to 100 fps.
For my testing I get the following results:
128x 10x fps drop
256 50-100x fps drop
Vvideo Example: http://www.youtube.com/watch?v=NVP5P4E3BJA
NPS File: https://dl.dropbox.com/u/84010144/1365052614347.nps
Crash Log Asked by Mod: https://dl.dropbox.com/u/84010144/crash_report.txt
Related issues
Attachments
Comments


Please force a crash by pressing F3 + C for 10 seconds while ingame and attach the crash report here.
This shows that TexImage2D must be rapidly recreating some of the textures over and over every single frame? CPU time is insane on it.
This is the crash reported requested by the mod for 10 seconds holding down F3 + C.
Thanks Kumasasa for the reply. I added the file for you though I don't think it will help much. Though I rewrote the bug report with new information I found while profiling the issue.
Pretty much it seems that if your menu is not open so your in the game playing, Minecraft is recreating textures for every single frame and or maybe tick. This has immense performance penalties when dealing with higher resolution textures.

Confirmed.
I'm wondering if this could be related to the 1.5 animated textures feature, could it be updating every icon on the texture sheet instead of just those that have changed ?
Apparently Optifine has managed to fix this issue. Come on Mojang, stop messing around with new content and fix this!
Grum has acknowledged this bug and stated they doubt they will bother fixing it for 1.5 and maybe be done in 1.6. I will not go into detail of the part where he thinks we should not use larger textures or get a better computer. I clearly think he has little knowledge on the situation. If someone can check out what Optfine fixed with it maybe we can add it into forge.
Kane Hart extremely disappointing that Mojang can improve support for custom textures, but then wash their hands of bugs that cripple the ability to make use of it. I too have found Grum to be useless at dealing with CPU usage/frame rate issues in Minecraft.
Forge has enough pull requests to sort through, without having a huge one to fix yet another Mojang bug. Previous attempts to put Optifine into Forge have been blocked anyway, and if the changes to fix this issue are extensive they could end up blocked as well.
@Paul Aslin. Problem is not even custom textures. Just normal textures do this too just very little and less impact on good PC's. The people who suffer a lot are also people who use mods. The big issue here is modders add more and more texture the sheet grows and the impact is also greater.
I managed to do a quick fix in Forge to fix the issue.
https://github.com/MinecraftForge/MinecraftForge/issues/513

Is this still a concern in the current Minecraft version? If so, please update the affected versions in order to best aid Mojang ensuring bugs are still valid in the latest releases/pre-releases.

See also MC-11875
MC 1.5.2 does fix this specific issue.
I agree, this could be tagged as fixed.
Still a lot of lag on 1.5.2. But only when scrolling left and right.