Entities stay on the global list even when the chunk unloaded

Relates to MC-142022 and MC-138871.

Confirmed for 19w09a.

Confirmed for 19w11b.

It seems that in latest snapshot either both commands give output or none of them does. Actual distance to teleport or walk before chunk unloads is bit higher than described though (>300 for chunk radius 8).

 
It still happens the same in 19w11b.
I think I described it badly, as one needs ignore the name in front in the outputs.
It will output the armor stand as long as it is on the global list, but only also have a second "Armor Stand" behind it if it is also on the local list.

When running the command:

/execute as @e[tag=MC141484] at @s run say @e[distance=..1]

The output before teleporting is:

[Armor Stand] Armor Stand

The output after teleporting is:

[Armor Stand]

That the name in front shows that is was found on the global list (aka @e[tag=MC141484]), while the name in the back shows that it is on the local list (aka at @s say @e[distance=..1]). Sorry for the confusion.
Thank you for looking into this!

@edit I replaced the commands in the post to only have an output if it is on the list. So now there is really no output for the command when the entity isn't on both lists.

@boq This ticket shows that entities can now be accessed under certain distance conditions, even if they are located in a chunk which is currently unloaded. For instance, you can access the entity, but not the block it is standing in, because the chunk is not loaded. Is that intended?

It is marked as confirmed and has a dev comment, so I would assume it's not intended. It's also inconsistent, you can do some things with those entities, but not many others.

Just for clarity, chunk with armor stand was not unloaded (it's border one), but function to query for all entities was missing check if containing chunk was ticking.

Hmm.. Isn't the non-ticked border supposed to be two chunks wide?
Also when I tried accessing the blocks (e.g. setblock) in the chunk where the armorstand was I would get a message saying "That position is not loaded" while the armorstand was still found via the global list.
You guys changed a lot about chunkloading recently so maybe my understanding of what's a border chunk is a bit off. Is there a new way of non-ticked chunks that are also not accessible?

Either way for 19w12b I can confirm this issue fixed, thank you 🙂

That required bit more explanation, so I made some graphs (using debug method obfuscated in 19w12b as 'void uq.a(String)', which dumps information about loaded chunks to file). Note that it is implementation detail and subject to change.

Every loaded chunk has few values associated with them:

• Ticket level - tickets are added on chunks that have to be kept loaded (like spawn chunks or chunks observed by player). Chunks without levels take minimum value from neighbors + 1. Player tickets have value 31, forced chunks have 32, spawn has 22, etc).

• Chunk status - more or less worldgen phase. Every status requires certain number of chunks of lower status around it. This status is stored on disk and can never decrement.

• Full chunk status - ticking status (entity ticking, ticking or border). It's called this way since only chunks with chunk status 'full' can have this value, as opposed to protochunks (though having 'full' chunk status does not imply this chunk is "full"). They are categorized based on ticket level, not persistent and can change during runtime. Chunks with this status have entities and block entities loaded (including global lists). Once chunk becomes "full" it never goes back to protochunk (but after unload it will be loaded back as protochunk and promoted in necessary).

When requesting chunk with given status, appropriate ticket level is required - for example, accessing chunk with status 'full' requires ticket level 33 or lower (if loading or generation is allowed, game will add 'unknown' ticket and provide neighboring chunks with lower status/higher ticket levels). Querying chunk for entities or blocks requires full chunks, so that explains why search failed (after teleporting, chunk with armor stand has ticket level 36, even though it already has 'full' chunk status - it's one way relationship).

On loading level values look like this:

That yellow "archipelago" to left is due to chunk being saved as full on disk. That does not mean chunk is queryable or that it's "full chunk", since that still depends on ticket level (which is not stored). You can also see that general ticking structure remains unchanged from previous versions.

Now, after teleporting to armor stand and back, loaded chunks look like this:

Now there is rather big purple area on third plot. Dump categorizes this as "border chunk" - they were loaded before, so they are stored in-memory as full chunks (algorithm never downgrades full chunks to non-full ones) and "border" is lowest full chunk status. So entities from that are still kept in global list (and in internal chunk list), but they are not accessible due to ticket level.

So simplest solution was to filter entities based on ticket level of chunk they are in. Though it seems I made mistake and required entity ticking level (31) instead of just border chunk (33), so chunks loaded with forceload (ticket level 32) are not accessible right now (partially MC-146470).

Note: chunk are dropped from list on ticket level 44. That's also when entities are removed from lists.

@unknown Thank you very much for taking the time to explain it!
We are currently trying to wrap our head around it and currently conduct a few tests to get a clearer picture 🙂
In any case, have a nice weekend and thank you again for your effort!

@unknown Awesome! This explanation helps so much to get a better idea of how it works.
Not that I fully understand it yet, but many things we observed in the snapshots make more sense now.

I had situations where a 5x5 of chunks kept being accessible after moving back and forth over a chunk border with server lag. Probably the ticket level of 31 of the player was not reset for a chunk at the border of the area the players ticket value is applied to.
The command block tool to visualize the accessibility of chunks broke two snapshots ago for reasons I have yet to figure out, so I didn't test this in the latest snapshots yet.

Rephrased from my current understanding:

• Chunk status = how far is the chunks generation

• Ticket level = value to determine what is required of the chunk based on what neighbouring chunks require of it

• Full chunk status = For all fully generated chunks, a classification based on the ticket level to determine the kind of processing done on this chunk: entity ticking, just ticking or border (the last one basically being "keep in memory" only)
  One thing I don't get about the graphics you posted is with the full chunk status. The red border (I assume that is the full chunk status "ticking") is only one chunk thick. But in game I still observe a two chunk thick border where entities don't get ticked but scheduled ticks are processed.

During testing I also did observe that I'm not able to access entities in the chunks without entity processing around the player loaded chunks. That's probably also because of what you described about the required ticking level for the entity list filter being off by a bit.

There is a lot more details I can't completely get my head around yet. It will surely take a while to become familiar with this new system.
Again thank you for taking the time to explain it here in such detail, that helps a lot! 🙂

Bit more explanation on ticket levels.

They are now core of chunk loading - they determine which chunks are loaded, how far in generation they should be and what full chunk (i.e. ticking) status they have.

Ticket level is determined based on level of neighboring chunks. Every chunk can also have tickets attached that set upper limit to ticket level of chunk (there can be multiple ones per chunk, but usually we only care about ticket with lowest level).

In other words: every loaded chunk has ticket level and zero or more tickets. Tickets are "sources" of ticket values, which spread to neighboring chunks.

Few key values:

• 44 - Last loaded level, everything higher will be unloaded. So ticket of level n will load square with radius of 44-n (i.e. (44-n + 1)² chunks).

• 34 - Lowest level that requires chunk to be fully generated and promoted from protochunk to full chunk.

• 33 - Full chunk with this or higher level will be classified as border.

• 32 - Level for ticking chunks.

• 31 - Lowest entity ticking chunk level.

• 22 - Spawn ticket level. It will load everything in radius of 44-22=22 chunks, i.e. (2*22 + 1)²=2025 chunk square.

Here is list of ticket level to chunk and full chunk statuses:

So now for examples:

Freshly loaded flat world:

One spawn ticket, few player tickets from view radius 8 and few 'unknowns' from random ticks (they are not permanent and will be removed on next tick). Note that even though chunk may have player ticket with level 31, actual ticket level might be lower if there are lower tickets nearby.

Freshly loaded flat world with sandstone on top:

Interesting thing happens around levels 35-38 - there are fully generated chunks, but still in protochunk form (no ticking status). During worldgen game tries to find proper spawn position. Since all surface blocks are sandstone, algorithm probes every chunk in square with radius of 16. Testing for blocks requires fully generated chunks, but after reload ticket level is not enough to promote to full chunk. So ticket level determines generation status, but not the other way.

Forced chunk

Single ticket of level 32 + neighboring generation.

Now, couple other things:

• updates to ticket level are done in "idle" phase of tick (when main tick runs under 50ms), so they may not happen immediately when server is overloaded. That's why some chunks may stay loaded even if player has moved.

• player has multiple tickets instead of one since view area may not be square due to move prediction

• I'm not author of this system, so this is just my understanding, not specification.

• As for non-ticking border - yes, red is (block-) ticking. Though entity ticking right now also has extra requirement of distance to player, so this may chance things. I would like to see reproduction steps for this.

Things start to get more clear. Especially

helps a lot 🙂

Paraphrasing again to avoid misunderstandings:
So some actions can place tickets on chunks to force them to load, generate (different levels) or load the surroundings.
You named a few things that can place tickets and what ticket level they have, like players and spawn chunks. I assume random ticks and block updates probably both cause (temporary) ticket level 33, judging by the level of the "unknown"s?

And then there is actions that only run if there is a certain ticket level in the chunk.
What I currently got from testing:

In a few weeks I will have time to test these things in more detail.

To the couple of other things

• The chunks that kept their ticket level when moving around with server lag kept that level even after the lag was gone. Only "picking them up" again would remove it. I will try to recreate it when I got time and make a report if it still happens.

• I was about to ask why the player doesn't have just one ticket, thanks for clarifying 🙂 (Also I think the player had one ticket until one of the first snapshots of this year, as this probably caused MC-141483)

• Haha, I would have taken you for the author after giving that detailed explanation. Just out of curiosity, may I ask who the author is?

About the 2 wide ticking border
After reading this second part of the explanation, I'm pretty sure what I described is due to ticket level 33 (border) is being handled differently than ticket level 34 (border).
This means MC-141482 is the same behaviour as the 2 wide border I described for the chunks around player placed tickets.
It's about /forceload "loading" more chunks than specified. Forceloading a chunk currently makes it's neighbours accessible as well.
Also the list of required levels above pretty much describes the observed already. Most things require <=33 (so a level classified as "border") rather than <=32 (which would be "ticking").
To test this behaviour:
1. Forceload a chunk
2. Take note of coordinates of any block in a chunk next to the forceloaded chunk.
3. Get far enough away for it to not be affected by the players ticket level
4. Run this command on the coordinates you took note of:

/execute unless block <x> <y> <z> air{N:O} run say chunk is loaded

The command only has an output if the blocks in the chunk can be accessed.
It will produce an output for chunks next to a forceloaded chunk.
Obviously it will also produce an output for chunks next to the "ticking" chunks caused by a players tickets.

Alternatively you can place a repeating command block with a say command in the chunk. That only runs if scheduled ticks get processed, so if you still get outputs that indicates that ticket level 33 still runs scheduled ticks. (and it does)

I you are looking at chunk loading logic and entity lists these two issues might also interest you:

• MC-141485 (change in size of spawn chunks because of the recent chunk loading changes, may be WAI)

• MC-108469 (age old issue about chunk wise entity lists not getting updated correctly, reproduction is partially broken in 19w12b due to MC-146470, see latest comment)

It seems that double BORDER layer may not be correct and caused by "magic value" at one point was changed from 32 to 33 - some places might not be updated. Anyway, values posted here are probably not final.

Also, right now only requirement for block entity ticking is level 33, so yes, inner layer of border chunks will tick. This is also probably a bug. Though it seems that random tick and other things work only on level 34.

I'm not sure if `/execute unless block` is proper test for ticking chunk, command block method seems like better way of checking for it.

And yes, unknown tickets are from random ticks, or more specifically, any access to chunk (players also create one when checking for blocks around, but only lowest ticket is shown in dump).

Fair point, the `/execute unless block` only detects if commands got the proper clearance for the chunk, which may or may not be the same as ticking.
I mentioned the command first, because this is what I used to query and display the status of chunks in game (which is how I noticed it in the first place). The command block method doesn't work well for that, as the block already needs to be in the chunk.
A `/getticketlevel <x> <z>` would be really useful for debugging *hint**hint* 😛

A /getticketlevel or similar would be awesome!
That'd be so extremely helpful.

Thank you to both of you for additions/explanations 🙂

For anyone interested in internals of chunk loading, `/debug report` in 1.14.4-pre produces tables similar to ones attached here.