Not Responding

When playing versions 1.7.10 and 1.8, 4 out of 5 times when Alt+Tabbing/focusing on other windows results in the game being flagged as "Not Responding" when coming back to the game window on Windows 7. This happens on both vanilla and modded. Allocating more memory doesn't seem to fix the problem, and having F3 open when this occurs shows that the game is still ticking and updating things. I also have the latest graphics drivers and the latest version of Java 7. Occasionally the game will start to respond again, but most of the times this occurs the only way to fix this that I have found is to force-close the game and restart it. This may be the wrong place for this issue, but I'm not sure where else to look.