When spectator's target changes dimension, the spectator is left behind in unloaded chunks.
Video demonstrates one account spectating another account. The target enters an end portal, and the spectator is left behind, with no chunks loaded. Works the same for nether, as well.
This is being tracked in MC-46421. Your video would be very helpful there 🙂