Entities duplicate and disappear

Please attach the crash report ([minecraft|http://hopper.minecraft.net/help/finding-minecraft-data-folder]/crash-reports/crash-<DATE>-client.txt), otherwise we can not diagnose this issue.
For technical support please use the Mojang Support Center.

-- I am a bot. This action was performed automagically!

What server software are you running?

-- System Details ---- System Details --Details: Minecraft Version: 18w44a Operating System: Linux (amd64) version 4.17.4-1.el7.elrepo.x86_64 Java Version: 1.8.0_171, Oracle Corporation Java VM Version: OpenJDK 64-Bit Server VM (mixed mode), Oracle Corporation Memory: 902773656 bytes (860 MB) / 2822766592 bytes (2692 MB) up to 2863661056 bytes (2731 MB) JVM Flags: 2 total; -Xmx3072M -Xms128M Player Count: 9 / 2018; [uc['DrexHD'/5102, l='world', x=25.04, y=16.00, z=27.42], uc['Hexo_mc'/7615, l='world', x=-80817.04, y=58.50, z=-21692.29], uc['_N0VA__'/10680, l='world', x=63.50, y=66.00, z=37.50], uc['wolfie_fan'/14912, l='world', x=-21259.84, y=81.00, z=-41994.52], uc['SnakeHunt3r'/15526, l='world', x=-80818.68, y=60.00, z=-21691.59], uc['davidtaco999'/15528, l='world', x=41046.39, y=58.00, z=-82909.69], uc['CaptainFireBall'/15886, l='world', x=83377.97, y=72.00, z=-26505.08], uc['BlueMangoBango'/17282, l='world', x=-80132.82, y=38.00, z=3199.92], uc['ReAnimatedEnder9'/16652, l='world', x=83377.50, y=72.10, z=-26503.50]] Data Packs: vanilla, file/kilocraft (incompatible) Is Modded: Unknown (can't tell) Type: Dedicated Server (map_server.txt)

That shows a invalid datapack, maybe that datapack was providing the entities you are missing?

Confirmed for 18w45a.

https://www.youtube.com/watch?v=hFENjs5Twg0&t=595

The most important part of the video is from 9:55 to 10:32, but I ran other tests to identify the bug.

Might be related to https://bugs.mojang.com/browse/MC-138919 (and if so, applies for 18w46a).
Entities going through any Nether portal are duplicated, sometimes rendered or invisible, but are noticeable on the console with "keeping entities with same uuid" spam. Those entities are either not responding to /kill command, or when they are, once they die, if you reload the chunks by going back and worth from the Nether for example, they will simply reappear. You can again try to kill them, but they will reappear once the chunks are loaded again, and so on.

Confirmed for 18w50a.

This is a very critical bug and cannot be left unchecked before the release of 1.14. This issue can leave a single entity to be duplicated thousands of times (log file in the dropbox link). There's more information on what this bug does to the world while the player is on it in this video, but I suggest you read more of the description than watching the video since it's almost an hour long. The log file encompasses the whole video just in case someone wants a reference:

https://youtu.be/SvyuYeoG6VI

https://www.dropbox.com/s/1fbvxe5hlbyns2q/latest.log?dl=0 (I used notepad ++ to see how many times a UUID was listed and I got a number over 6 thousand on the first attempt)

This duping entities issue is related to chunks loading. Marking the chunks on forceloading makes the entities not duping anymore (but brings other issues by itself).

Confirmed for 19w02a

This might also be related to MC-141484.

Can the clones be found with a global entity selector?
What I mean by that, is a selector which does not contain a distance. Because when you use the distance in a selector, the game will usually use the chunks entity list, rather then the global one.

@unknown Clone entities are tracked on the global entity list, but not the local one.

@unknown Thank your for confirming 🙂
So yeah MC-141484 is related, MC-142022 as well.

They all describe situations where entities are only on the global list and issues that likely result from this.

Having two versions of a villager with a custom name, where one is a clone of the other (same UUID), when unloading the chunks where they are, one version of them is still tracked on the global entity list. If I come back and load the chunks again, the two of them are tracked on the global list, but only one of them is tracked on the local list.

Confirmed for 19w03b.

Confirmed for 19w04b.

Confirmed for 19w05a.

Confirmed for 19w07a

Confirmed for 19w08a.

Confirmed for 19w09a.

To those that can confirm this, can you please provide steps to reproduce?

The comment I made on January 3 is very useful for this.

Since 19w09a (or possibly 08), it is inconsistent to reproduce but it was still possible. I tried many times with a new map, and it only happened once, although it produced more warnings (see below).
Some versions ago, I had a trading hall outside of spawn chunk, with one villager per "box". Once I relogged in that area, and upon relogging, I had now 2 villagers on almost every box, in a sitation similar to this:

, one was the exact clone of the other.

It seems that the entity tracking is getting better the more versions are released. This is how you can try to reproduce it:
Go outside of spawn chunks, build a Nether Portal. Enter the Nether, exit the Nether, enter the Nether. Repeat and watch the game logs. The cause of the bug is still in this version, because sometimes, you will see warnings saying "keeping entity with same uuid." In previous snapshot, when I tped to these entity, it was a clone of another one. Now if I try to tp, I got an error saying entity not found.
Still, I was able with this version and a new map by entering/leaving the Nether several times, and outside of the spawn chunks (this never happens if the chunks are loaded), to dupe a chest minecart. I got this warning:

[23:17:42] [Server thread/WARN]: Keeping entity minecraft:chest_minecart that already exists with UUID 52bafaf2-894d-4eb3-b4ad-322f8c4ab874

Then I tped to this entity by uuid. And I found two minecarts stuck inside each others. There is a screenshot, although I moved one minecart away from the second one when I tped to them.

Then I hit the /kill by uuid command to kill the minecart, but only one of them was killed (one left). Then I relogged with second one still intact. And upon relogging, the second one vanished.

Another note that this issue has evolved between versions. With late 2018 snapshots, I could kill the duped entity many times in game, every time I relog, it would reappear. By reading the region file, there was still mention of the duping entity and the only way of removing it was to hard edit the region file.

Now with this minecart issue, I read the chunks data inside the region file, and after killing the minecart, there was no more mention of the second one.

I can confirm changes have been made recently on this. I now tested it on a server. Usually, entities were duping upon either relogging, tping or dimension traveling in/to/between unloaded chunk(s). This doesn't seem to be the case anymore.

MC-144784 was possibly a sideway of recent changes on entity tracking and its fix might have "almost" fixed this issue?

Hi,

For me the issue happens 2 from 3 on multiplayer server. Several stop/restart can help to solve the problem.

My method to reproduce:

1. Join the server

2. Go to a village and find the fenced area usually found in plain villages. Or build a fence yourself.

3. Summon 10 cows in that area

4. Stop server and restart

5. Join again and you have 60-70% chance, that the keeping entity warning appear on command window of server. That means your entities are duplicated, twice as much cows in the area is noticeable. 🙂

Just for sure, you can summon 50 chickens too, in an other fenced area, because on my server this issue affects mostly chickens.

If you notice a sitting cat with 2 heads, you can be sure, it's duplicated.

Even the villagers are often duplicated. Advance their tiers with trade and you can notice the difference: the duplicates have always one less tier than the original.

In some cases unfortunately the original villagers remain invisible, just the duplicates appear.

Since I run the server on the same computer as the client, maybe that's a performance issue, but I'm not sure. However the screen loads much slower than in 1.13.

HTH

But @pluztig: did you test it with latest snapshot, in a non previously affected area?
Even if this bug is fixed, it does not mean your part of the world already affected will be. This won't happen on new area, but you'll have to remove the "old" duplicates yourself, and the game will still complain about "keeping entity with same uuid" that was caused by earlier versions.

@Johnibur that was a good question, so I tested it quickly. 🙂

I made a brand new server world with level-seed=5000000123456

I located the nearest village: (-460; 77; -930)

I summoned 10 cows and 100 chickens.

I disconnected/reconnected and the server already began to display the keeping entity for chickens, donkeys, llamas, creepers. etc.

I restarted the server, but the problem occured again.

Unfortunately the foxes killed many chickens, but cows are duplicated on 2. reconnect I think.

Here are my 10 cows:

I counted at least 12 on this picture, however many of them didn't move yet, so they are not "separated"...
 
As I mentioned, it happened on a brand new world on multiplayer server.

for the 2nd server log

HTH
 

@plusztig You really have a lot of warnings in your logs, that's something I cannot get myself while testing with this version. So it must be related to something else than just the version. What is your view distance? Can you post your server.properties file? I would like to try it myself by reproducing the same steps (same seed, same server config).

@@unknown Attached.

And the 1st server log, which already contains some keeping entity error after the teleport!

This "clickable" teleport coordinate function also needs some improvement, because I fell out of the world till I found the appropriate Y coordinate above the ground... 🙂

This bug is affected by game's performances and being outside of any loaded chunk. The more lag you have, the more entities will dupe.

I exactly reproduced your steps, generating this world with your view distance (16), then I tped to the village with all these different entities around, spawned cows on a fence, and relogged from different parts of the village.

With render distance of 16 as yours, I got a certain amount of entities duping, inside the village and outside, and the cows duped. But the number of duped I could see and the warnings I got are less than you could get. Also my world generated twice faster and it didn't crash. Those are the logs:

Then I deleted the world and I redid exactly the same process, but with the normal render distance of 10. This time, no cows duped, and no entity duped inside the village. Only a few warnings about hostile mobs in caves, despite having a lot of entities around. Logs:

Finally, I did it again with render distance of 6. And at first I got no duped entities at all, then only a few again in caves. Then I tped to one of them to see. And upon relogging from higher ground, only one cat duped in the village. Logs:

So in conclusion, we can see that performances affect this issue. And it was tested on localhost. On a dedi server, I could barely find any duped. I think with further upcoming performances fixes, the issue could be fixed by itself. To be tested again with future versions.

@@unknown you're fantastic! 🙂
I hope they fix this bug soon. According to your measure this event occurs even with render distance=6.
We're used to play on usual business laptops. These are not gamer machines, but MC worked fine till this time. But now even on a dedicated (business laptop) server the new version is very slow.
I don't understand why the duplication is related to speed of server. I think duplication is not acceptable at all. Neither on a slow server, nor on a fast one. 🙂
Again thank you for your measurement, I hope developers can use these results to fix it. 🙂
Cheers,
Peter

Confirmed for 19w11a.

Is anyone able to get entities to duplicate?

I only see the "Keeping entity" spam in chunks where entities already duplicated and disappeared, but I can't tell if any more are duplicating.

I redid the same steps as above. It's now rarer than it has even been before to reproduce. Mostly you get warnings about "keeping entity with same uuid", but couldn't observe dupes. But I was still able to observe one dupe but only under special conditions (high view distance, and relogging several times).

I just spent about an hour in 19w11b killing duplicates from my world (created in 19w09a). I suspected that view distance was responsible for creating them (had it set from default 10 to 16), so I knocked it down to 8 and have not had more duplicates spawn since. Not sure how prevalent this bug was before the 1.14 snapshots (never experienced it prior), but now it seems like only one entity with the same UUID is displayed in the game at a time, chosen seemingly randomly. After killing a duplicate entity, another with the same UUID will be loaded upon relogging. Deleting the duplicates took a very long time as some entities in my base (mostly just specific ones) were duplicated upwards of 15 times, meaning I had to relog at least 15 times to get each duplicate to show in game.

Duplicate entities that are invisible still interact with the world. They are immobile and prevent placing of blocks at their location. Visible mobs such as villagers and golems often get hung up on them when trying to path and sometimes even "look at" them even though it appears nothing is there. There were about 200-300 invisible duplicate entities in my base, to the point where I noticed a lot less lag once all were dead. This is actually a pretty serious problem.

If Minecraft is able to detect these duplicates, a temporary solution should be to automatically delete them. I do not see the harm in doing so. My world is fine now after manually removing them.

Can someone upload a screenshot with entity count in F3 debug screen?

Is this reproducible with a freshly created world in the latest snapshot (19w11b)?

Yes, that's what I did last time, trying on a new map. It is now rarer to reproduce, but it was still possible. Basically, depending on your configuration, with normal view distance you can have from no problem at all, to warnings saying "keeping entity with same uuid". In earlier snapshots, we observed this warning was meant to entities being duplicated and it happened all the time with every entity outside of Spawn chunks. Now the warnings can appear on a new map (outside of spawn chunks) when you relog, or go to the nether, but generally under more "extreme" conditions (view distance larger, more pressure on the server). We also correlated that people having a weaker configuration can have this issue more often than others (but this correlation was made in 90a).

Personally, with a decent configuration (8 x 3.5GHz, 4GB allocated), that's what I did to reproduce in 11b:
Creating a new map, with view distance of 16 (with less it was difficult to reproduce, and correlatively saying less message "can't keep up, is the server overloaded?"), locating a village outside of spawn chunks, going there, and relogging there several times (making the chunks unload / reload). Got a few "keeping entities with same uuid" warnings. Then after going to the Nether back and worth several times, finally seen a duplicated sheep (One were inside of the other) and having warning about "keeping entity sheep with same uuid". Then I killed the sheep by imputing the incriminating uuid, which occurs to have one of these sheeps being killed, but the other was still there. Then I relogged. And upon relogging, the other sheep was gone too.

I haven't created a fresh map but 19w11b still definitely a large scale issue with my villagers. Each time I start my server I have basically two sets of villager employees, each server restart I either get my master shepherd, fisherman, nitwit, and librarian, or on other server restarts I get a regular shepherd, farmer, nitwit, and librarian. They have different trades, I think they're duplicated and half go invisible when the other half appear.

If you had duplicates on the same world before I doubt any fix will clean your entity list. You'll have to clean it by yourself or start a new world.

I've done extensive testing with the same setup I used in 19w09a. Then I easily got the "Keeping entity with UUID" message in game output and a lot of both invisible and visible duplications to the point where the game barely was able to load because of all the entities created through deaths, relogs, teleportation and moving between dimensions. In 19w12b I was unable to recreate the bug at all even though I really stress tested with many more entities causing much more lag towards the end of the session. I think MC-141484 being fixed might have solved this one.

I would be happy if I could not reproduce it. Just yesterday I tried to kill all duplicated entities from earlier version, one by one. Whan all of the sudden all of my already checked and dupe-killed chickens were duplicated. Each of them, all of the 20 pieces.
However I noticed that much less duplicates appear in the recent versions, but it can! 😞
Maybe because I hatched them from egg earlier? Or why only the chickens? Why not the cows, pigs or wolves? I don't know.
I killed all of them, and summoned 20 new.

Peter Lusztig, to be clear my testing was done in newly created worlds in the latest snapshot each time I tested this. I used structure blocks to get the exact same testing situation and locations in the different worlds, but the testing was done in fresh worlds. My server world that has passed through all the snapshots, except the crashing ones, is beyond rescue because of all the duplicates though. I can command-kill every single duplicated UUID in an area only to have them return when I relog. This of course sucks, but can in no way be Mojang's responsibility, since I chose to use snapshots on my server fully knowing the risks that this entails. If the bug is no longer reproducible in a fresh world in the latest snapshot it is most likely fixed. If you on the other hand can reproduce it in a fresh world it would be valuable to know exactly what steps you took to reproduce it so others can give it a try as well. 

@@unknown: unfortunately it's very easy to reproduce in fresh new world, with the latest snapshot, too. I mentioned earlier in an other ticket or in this one, I don't know. 🙂
5 minutes of playing is more than enough...

Lets see:

1. I created a fresh new multiplayer server world with the seed: 5000000123456

   • I played with view distance = 16, here is the server.properties

1. I located the nearest village and teleported myself to there: [-463.5, 80.0, -943.5]

   • unfortunately I died at first, because the game loaded the chunks too slow and I I fell out from the world or similar...

2. Again I tped myself to the village

3. I summoned 10 cows and 30 chickens

4. I disconnected and reconnected

   • here I got 3 keeping entity messages

5. I began to investigate these, so I tped myself to the dupes entities

6. I disconnected and reconnected

   • here I got a lot of keeping entity messages related to chickens

7. I began to investigate these, so I tped myself to the dupes entities

   • A lots of keeping entity messages appeared this time for pigs and wolves

I attach the server.log:

However the server display only one instance of these entities above, these entities are already duplicated: e.g. lamas spit at the "invisible" wolves, they bark, etc.

Please try to reproduce it. It's quite easy. 🙂
It turned out from our previous measurements that maybe the server speed has some relation to duplications, I mean more lags means more dupes.

HTH,
Peter

@unknown could you send me the world that you used in your test that produced the `latest.log` file? If it's not too large you should be able to pack the world (zip) and attach it to this bugreport.

As we have discussed here since 09a, we have noticed the bug depends on your specification. Under the same circumstances and same server properties, reproducing the same steps, one can produce more duplicate on their world than another if not at all.
On this version, I have followed the same steps as Peter again, and I'm unable to reproduce too (same world, same properties file). I'm on 8x3.5GHz with 2GB allocated.
@unknown What's your specifications?

I recently learned that chunk loading/unloading is done in the idle time of the tick. As this issue probably happens on loading or unloading, it is likely that server lag has an influence on it.
Also @unknown writes about server lag and in their

there are quite a few "Can't keep up! Is the server overloaded?" messages.
@unknown If you got time for it, you could try to recreate it on your system by artificially causing server lag. A good way to do this is to set the random tick speed really high (e.g. /gamerule randomTickSpeed 10000000; good value needs to be found by experiment) and place at least one block that receives random ticks around the player (I usually use vines).
If it was possible to recreate it with this method, that would indicate that it's not so much the system that's important, but rather how much load is on it.
Hope this helps 🙂

I just noticed MC-108469 is not in the related issues, while it's an understood issue about disappearing entities. I'm not sure how closely the relation is, but it should definetly be in there.

Hi @@unknown,

It's a fresh new world, so it's not big (18MB), but max. 10MB is attachable here.
So here you can download:
https://www.dropbox.com/s/thpbgc6dcts5gc6/Test_Experimental.zip?dl=0
I hope it works, because I already deleted it before. Just I restored from Recycle bin. 🙂

@@unknown: I have a simple business laptop
Dell Latitude E5470
Processor: Intel(R) Core(TM) i5-6300U CPU @ 2.40GHz (4 CPUs), ~2.5GHz
Memory: 8192MB RAM
Video: Intel(R) HD Graphics 520

It's not a gamer machine, I know. 🙂

I was unable to reproduce by creating artificial lag like @unknown suggested. What happens if I set the random tick speed to 10,000 is the lag is such that the chunks are not loading (or being sent) but notice the entities are present (in the "void"):

If I increase the speed, eventually the server will crash upon exceeding maximum tick speed.
I didn't test lower values, it was just mainly rhetoric. If someone wants to tackle it, go ahead.

Yesterday evening I tested with the newest snapshot. I set view-distance to 10, so I tested it with deafult settings.

It was a bit more difficult to reproduce the bug. But it still exists!

The scenario and the seed was the same: fresh new multiplayer server world with the seed above. I had to wander around 10 minutes with many disconnects/reconnects, but finally 2 wolves were duplicated.

Latest log attached:

Worlds with duplicated entities loaded in 1.14 Pre-Release 5 will have the entities deleted as long as the chunks that they're in are loaded one last time. The "Keeping entity" spam will appear in chat, but no more afterwards. Example