Unloading chunks when they shouldn't unload

In some places chunks unload. But they shouldn't. How to reproduce:

1. Find a side of a chunk.
2. Go into the new chunk with F5 mode (back).