Solved

I just tested this, they do respond, but the game lags extremely and that creates the illusion that nothing is happening. As soon as the game actually finishes computing a game tick, it grows.