Frustum culling does not recalculate after changing perspective

If I look at an object that is >= 2 chucks away from me, switch to 2nd person view, move the mouse, and go back to first person, the object I was looking at is no longer visible. Once I move the mouse, it comes back in to sight. The issue seems to be that the frustum culling does not recalculate when changing perspective, but it does recalculate when you move the mouse.