mojira.dev
MC-118106

High CPU usage, lag and poor performance on dedicated / integrated servers

Multiplayer / Integrated servers have very poor performace and cannot keep up even with one player on a high-end PCs / Servers / Machines

Description

Vanilla server platforms (integrated and external) are very poorly optimized and can barely run smoothly, even on a high-end hardware with higher amounts of memory allocated.

Typical, small multiplayer server drops to critically low TPS and can crash / be forcibly shut down by the watchdog (if enabled) with just a few players normally generating new chunks (in survival gamemode).

A server, using the "Default GUI" is almost impossible to run stable. When launching it via command line and allocating sufficient amount of RAM, the performance is a little bit better (and the server is not running out of memory), but the playing experience is still very rough and there is quite a lot of rubber-banding, freezes, lag spikes, etc.

How to reproduce the issues & Testing methods
Singleplayer (Integrated Server)
  • Launch the game with plenty of memory allocated (4GB is usually more than enough)

  • Enter into a world or create a new one

  • If creating a new world, initially there are extreme framerate drops (probably client-side issue), the integrated server is rubber-banding and skipping ticks

  • Start exploring the terrain around and perform the usual things (generating / loading new chunks, dropping item entities, etc.) - There will be noticable stuttering, lag and rubber-banding when punching mobs or making redstone circuits, for example

  • If the player is in creative mode (or spectator mode) then everything becomes much worse

If you are playing in a survival world for long enough (assuming there are animal pens / farms, some mob grinders, chest rooms with item frames and similar) the world will run much slower and this can cause issues with redstone circuits, influence on the gameplay and so on.

Multiplayer (Vanilla platform, without any mods / tweaks / changes)
  • Start a server on a relatively powerful hardware with sufficient amount of memory allocated (4GB again, would be probably more than enough)

  • Join the server, along with a few other players (2-4 players at the same time is normal, 5+ will be a bit heavier)

  • Start playing as usual (exploring around, interacting with entities, mobs, items) - There will be clearly noticable rubber-banding and lag spikes, which usually result in stuttering entities, desyncing issues between the clients and the server and other

If some of the players start to generate new chunks on creative mode, the server will most likely crash.

Testing methods & RAM allocation

Tested and experimented with many combinations of JVM heap sizes (amount of memory allocated), using the default, "recommended" start-up command line:

java -Xms<Initial Memory>M -Xmx<Max Memory>M -jar <Server Jar>.jar nogui

(Xms = Starting Memory / Xmx = Maximum Memory)

  • Xms1024M / Xmx1024M

  • Xms2048M / Xmx2048M

  • Xms3072M / Xmx3072M

  • Xms4096M / Xms4096M

  • Xms6144M / Xmx6144M

  • Xms8192M / Xmx8192M

  • Xms1024M / Xmx2048M

  • Xms1024M / Xmx3072M

  • Xms1024M / Xmx4096M

  • Xms1024M / Xmx6144M

  • Xms1024M / Xmx8192M

  • Players at the same time: 1-3, 3-5, 5-10

Usual activities during the tests:
  • Exploring the terrain around in survival mode, creative mode (for a while), spectator mode (just for the experiments, not expecting actual results)

  • Interacting with mobs, entities, armor stands, droppping and picking up item entities

  • Placing water and lava (creates many block updates)

  • Small, medium and advanced redstone circuits (light, piston, hopper updates)

  • Small, medium and large mob farms / grinders

  • Updating gravity-affected blocks (sand, gravel, concrete powder)

  • Entering different dimensions and teleporting items, mobs, entities through portals

  • Transfering small and large amounts of items through hoppers

  • Loading previously generated and alredy saved chunks

The RAM & CPU usage (especially) was abnormally high for very small and short activities or sometimes none at all. Not only chunk generation is causing the more serious issues (but mainly it is).

Conclusion

The main reason for most of the issues is not exactly the TPS drop / loss / poor server performance, but client-side ticking, updates and synchronization.
Other games (clients) handle the updates, coming from the server in different way.
If the server can't keep up with the consistend update rate / tick rate, then the client should enable some sort of "interpolation" and increase slightly the delay between the changes (updates).

The Minecraft client runs at hardcoded 20 TPS and waits for the next tick from the server. If it is synchronized (matches with the incoming client update / tick) then it runs normally.
But even if the server drops to 19.99 TPS - There is desynchronization once in a while and this can create issues (mainly client-side).

Overall, the server performance should be definetly optimized more and some things can be tweaked.

Related issues: MC-117611, MC-44385MC-4890MC-342MC-54026

Related issues

Attachments

Comments

migrated
[media][media][media][media]
[Mod] Neko

Does MC-117611 describe your issue?

migrated

I've noticed the same problem, but in singleplayer.

migrated

when in singleplayer, and on server I also experience the dreadful lag even on a i7-7700HQ turbo-clocked

migrated

@unknown I see you have 3 machines listed/tested but just to clarify:

  1. What is the OS the servers is running on?

  2. Is your server running on a dedicated machine and client on another?

  3. Is your server running within a virtual machine?

  4. Is your server running any other virtual machines?

migrated

@unknown Thanks for pointing this out, I forgot to clarify these particular things.

All machines are running the latest Windows 10 (x64) all the time. However, along with the testing I had several OS re-installs on two of them. (The second one in the "environment" field is my friend's PC, we ran multiple tests not long ago)

My main and "secondary" PCs (the 1st & 3rd) were running on Windows 7 (x64) quite a while ago. I had initially performed the first tests back then when I created this issue.

I've tested multiple operating systems, including Windows 7, Linux Ubuntu (Server & Desktop edittions), Windows 10 (the older versions) and currently the latest versions / updates.

None of the physical machines was / is running Virtual Machine in the background, nor other external softwares / background processes.
The client was running on a different device (laptop) and the connection between the server(s) and the client(s) had no effect in any way while testing.

Additional info:
I, along with friends of mine can confirm the issues, we were experiencing. During the tests there were other, actual players for more realistic and accurate results. (Not idle / AFK clients, just connected from one place)

migrated

https://bugs.mojang.com/browse/MC-117611?focusedCommentId=458596&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-458596

 

Reply to Neko: If this is now an issue again, the status on MC-117611 by logic, is re-opened?

migrated

@unknown I think MC-117611 is still valid, but it describes high tick delay and extreme lag (which is still present currently in 1.13-pre1).

This ticket (MC-118106) also describes high resource usage (CPU & RAM) in more detail, along with the very bad lag and high tick processing time.

migrated

I get big lag spikes even when generating empty chunks or playing on superflat worlds.

migrated

Affects 1.13-pre7

migrated

Still an issue in 1.13-pre10.

I have a single server running, with one player logged on, on an entire dedicated box - Intel i7-7700k @ 5GHz, 64GB RAM and NVMe storage - and I'm seeing the same issues.

migrated

Confirmed for release 1.13.

Until this bug is fixed, rather than being a simple boon, pregeneration will be a mandate for all (even small) survival server owners who want their players to have a decent experience.

In previous releases, pregeneration was nice, but I hardly did it because my servers were mainly for a few people and we experienced no lag at all.

migrated

Affects 18w31a, TPS still decreases to about 8-9 when generating new chunks.

Asteraoth

It seems like 17w47b is the 1.13 snapshot that added extreme lag from exploding TNT (MC-135529 and MC-133867). Exploding TNT in snapshots 17w47a and below is the same as 1.12.2.

migrated

Confirmed for 18w33a.

migrated

Still Unassigned in more than a year?
Confirmed in EVERY 1.13 snapshot.
Playing 1.13 on a multi player survival server is near impossible. The tick lag / client server sync is so far off it is near unplayable. Example: Bridging over a ravine, the void, the nether, you place a block, you see the block, you move to said block... the block is now gone and so are you... and all that you've worked for. Misplace a gravity block while in water and you take a ride down into the void. A hostile mob attacks me, I attack back, the hostile mob floats in the air while I perform a "Matrix" move on it. I'm not that OP. I watch videos were what once could be done now is near impossible, all the while playing on a respectable white listed SMP server experiencing the exact same issues. Missing items from player inventory, furnaces that only smelt some of the items yet burn the fuel for all of them. When the server restarts, the game is somewhat playable, and yet predictably becomes more and more unplayable in a short period of time.
I am sorry Mojang but I feel like I'm beta testing right now. I'm not sure how I can help, but this is really bad. Please help me help you.

migrated

i've noticed myself on my server machine running 18w46a that the server instantly pings the CPU at 100% on a brand new map regardless of how many players are on and eventually times out to watchdog

migrated

Yes, and the server starts to lose TPS fairly quickly after a restart.

migrated

100%, single thread, server on Ubuntu entirely unusable in this format. Tried all sorts of java arguments, no dice.

This needs fixing asap!

migrated

Watching the recent changelogs and bugs being fixed, but apparently this is not a priority, shame!

migrated

Confirmed, still seeing this behavior in 19w04b.  As a multiplayer (LAN Shared) client, to get the chunks to load, I have to disconnect and reconnect.  After that, as a client, it will forever be stuck on "Waiting for chunk".

ZeNico13

Still in 19w07a

migrated

Confirmed, still a problem in 19w07a.  I am running my multiplayer world on a lightly loaded Linux server as follows:

$ java -server -Xms2048M -Xmx8192M -jar server.jar nogui

With only two players it is hardly usable - we frequently have to reconnect to get the chunks to render.

This is on a new world , created with 19w07a.  We aren't using command blocks or redstone and have not spawned any mobs that didn't spawn on their own (and no breeding).

ZeNico13

Still in 19w08a

ZeNico13

Still a problem in 19w09a

ZeNico13

Still in 19w13a

ZeNico13

Still in 19w14a

migrated

My server network just made a 1.14 pre release 1 server and have noticed the same thing it will constantly post the error Can't keep up! Is the server overloaded? and it will be running as low as 30-50 behind as high as 500 behind. Updated today to pre release 2 and still having the same issues. Its causing my players to not want to play on vanilla minecraft as they say its badly optimized and not playable. Please get on this issue as soon as you can its game breaking at this point.

 

migrated

With 3 people on running Pre4 my RAM usage dosent go above 10.5% & CPU usage fluctuates between 34% to 45%

ZeNico13

Still in 1.14 pre-5

ZeNico13

Wow! Still in 1.14 Release! It's sooooo bad! We have the impression that we came back in 2018 with the first snapshots of the 1.14 with the new rendering engine!

How was it done to regress at this point? It was not too bad in Pre-Releases...

migrated

At least they fixed it from idling at 100%. I am noticing a lot of my 1.14 servers have horrible CPU waste.  A lot are idling around 20-30%, a few at 40-50%, and the rest (Most) at 15%.  But it takes SO LONG for it to get down to 15%.  And the Ram usage in 1.14 is horrible.  A Vanilla server needing 2GB of ram at idle is ridiculous.  I know Java Edition can't be milked dry for money like the other platforms, but they could at least care for it a bit more.  Or actually respond to us.  

 

Xeon E3 1270 V6 3.8Ghz

migrated

1.14 was very fine for my server, CPU usage stayed below 60% even with 10 players on, with next to no lag except for funny chunk loading.

Now in 1.14.1 Pre-Release 1, chunk loading is fine, but the server is clogging the entire CPU again and we're experiencing pretty bad entity and block lag with the same number of players. I'll watch the server when it's idle again.

migrated

It's slightly better for me in 1.14.1, but it still only takes about 10 minutes of play for it to overheat my computer, in singleplayer, with all the settings turned down. I never had these problems in 1.13. 

ZeNico13

It's better than before but still in 1.14.1 Release

migrated

I can see that the issue was created for version 1.12. I used to run the server on 1.12 but on version 1.14.1 server runs at least twice as slow. I'm using vultr and 1.12 server run just fine on 20$ subscription. Now I'm running 1.14.1 server and it's lagging and using 100% CPU on 40$ subscription. It really sucks...

migrated

Just twice as slow? Compared to 1.12? Lucky you!
My server runs three times slower, compared to 1.13.2...

Powerful server always running perfectly smooth on 1.13.2 - after upgrade to 1.14.1 with a new world generated the server can't keep up even with 10 players and with render distance lowered to 6 (was 15 on 1.13.2):

  • enormous resources usage, eating up to 16 GB of RAM in a matter of a couple of hours, even with server limited to 10 slots only;

  • CPU also enormously stressed most of the time;

  • basically, resources usage is at least about 3 times higher than it was on 1.13.2;

  • frequent crashes due to high resources usage, low tps, lags;

  • logging in by someone takes very long and is causing terrible lags on the server;

  • people are often getting timed-out kicks.

migrated

@@unknown
 
While this ticket (MC-118106) is indeed about performance issues, it is more about long standing structural issues. If you have experienced a massive change within 1.14 then it is more likely that you are experiencing both MC-138550 and MC-118106.
 
You really should look into MC-138550. It's an acute issue that has arisen during the 1.14 shapshots that has been partially resolved several times but it keeps returning with a vengeance. If MC-138550 does appear to be impacting you please show your support on that issue as we're not even sure if Mojang acknowledges it as a legitimate issue as we haven't seen any official communication as of yet.
 
I hope that helps.
 

migrated

The problem is less in 1.14 than it was in some earlier snapshots, but despite high resource usage the server still lags behind quite often and for a long time, sometimes seconds at a time. I still run 1.7.10 and older, which even with mods don't come close to the same resource usage nor experience the fairly extreme timeouts. With two players using the same connection to a server less than 400miles away on a completely barren map still use 60% of 800% total is not acceptable, especially when timeouts and lag are still present. The fact this issue is being completely ignored, while other completely trivial bugs seem to get more attention is disgusting, shame on you Mojang!

migrated

Amen!

migrated

Just discovered something: Disabling mob spawning removed all signs of the lag. At 40 players it performed exactly as it should on a $120/month machine.

What I mean is that entities should not have affected performance as much as they have, that is remove the problem completely. It's likely the performance leak is somewhere in there.

migrated

@MetoolDaddy

This confirms my opinion also.

Recently after many days of observations I discovered - and now I'm 100% positive about it - that one of major impacts of server-side performance is broken new AI of mobs, especially pillagers and patrols. My server keeps stable 20 TPS but when I meet a patrol TPS starts to drop and after wiping out the patrol it is raising again back to 20. I think this is something very critical that devs should look into to make the game playable again.

migrated

The performance drop did begin before 1.14 though. Maybe we're dealing with an entirely different issue by now?

This bug report might just be too broad.

migrated

@Kamil Pigłowski @MetoolDaddy Yes, I can confirm that disabling mob AI and/or spawning greatly improved the performance in certain scenarios like mob farms, large spaces with spawn-able platforms, etc. However, the server performance overall seems to have dipped way down in recent versions (1.13 / 1.14), compared to older stable versions like 1.11 and 1.12.

According to my observations and tests the major hit in performance, as expected, is still caused by chunk generation/loading, entities, mob spawning and AI.

This ticket is focused primarily on that, along with some additional info on how the gameplay is affected. I don't really know if the stated details cover too large area, but if other tickets contain all specific points that are listed here, maybe this one should be split and carried over?

To be honest, there are way too many small bits of details about performance issues in hundreds of tickets to keep track of, let alone filter and create separate report for every single one.

I'm open to suggestions on what could be changed here, or if the mods decide to close and split the ticket if needed.

migrated

So I shifted my world to Minecraft Realms and it was great at the beginning but after 3 weeks of playing it gets really laggy again. In terms of content versions 1.13 and 1.14 are great but it's almost unplayable on multiplayer servers

migrated

1.14.4 Survival SMP whitelisted server launched today. Roughly 48 players on and off. Within the first 2 hours the server had crashed at least 3 times. Rubber banding, block lag, near zero chunk loading, cannot eat, cannot pickup items. My dog would never teleport to me even. You can't travel by boat or horse since the server complains it moved to fast. This is insane and purely SUCKS! The SMP experience is being ignored in my opinion. While I'm on a rant, it's interesting that the 1.14.x lighting glitch seemed to go temp fixed/ignored until Hermitcraft tweeted about the issue.

migrated

@xavom

Unfortunately Mojang seem to think that everything's fine with the current state of the server; I certainly have not seen any acknowledgement from Mojang about the terrible state of affairs or any plan to investigate or resolve the issue. Mojang could be working on it, but if they are they aren't talking. I have seen community moderators asking for profile data, but there is no sign that the data is actually making it to anyone at Mojang. It's pretty much been radio silence. And, for the hosting community, this has had a massive impact, which makes the lack of communication even more hard to deal with.

Here's is Mojang's current solution to your problem: buy better hardware. For your needs (48 players SMP) all you need is a 1024 core 300Ghz Xeon CPU, 512TB of RAM and a 2TB/sec SSD. I am sure you can get one of those for cheap at your local PC store, right? It's not like their current published hardware recommendations are total garbage...

1.14 has been a disaster. It's generated a lot of renewed community interest, which has meant higher than usual server loads, which all would have been fine if 1.14 wasn't also a total dog in the performance stakes. For nearly a decade the hosting community was a huge part of Minecraft's success, but the double punch of Realms and Mojang's ongoing hostility to the hosting industry is making it very hard to provide servers for the community.

violine1101

Unfortunately Mojang seem to think that everything's fine with the current state of the server

Incorrect. Mojang is aware of the situation and planning for further performance improvements in 1.15.

So far I also have not seen a single ticket about a specific performance issue. So far everything I've seen was just "help my minecraft is laggy / has bad performance". That's not particularly helpful. Also, so far nobody (as far as I can tell) has posted a debug report that shows bad server performance, or what the bad server performance is coming from.

I want to add that this ticket at this moment of time is just "Server performance in MC 1.12 and above is bad". If someone could find out exactly why that is, the chances that performance can be improved soon is increased significantly.

migrated

There is hard to tell about a specific performance issues because the whole 1.14+ and still 1.14.4 is one big performance issue. There were tons of specific reports for example in https://bugs.mojang.com/browse/MC-138550 but the ticket has been closed. Besides, players who paid for the product are not your employees. Maybe hire your own quality control or beta testing team. Let this team just start a good-hardware server and make it for 10-20 people. Let that team play the game normally for a couple of days, explore, trade, breed, fly elytra, do some raids, just play the game normally and you will see how dramatically bad it is comparing to 1.13.2 (not even to 1.12.2). You will see for yourselves how many lag spikes freezing the game even for a couple of seconds there are, how many TPS drops below 10 there are, and how many general lags and shutters there are. Figure it out yourselves why is that, do reports on your own. When players are trying to help giving you most accurate info possible comming straight from their gameplay and attaching tons of reports like in https://bugs.mojang.com/browse/MC-138550 then the biggest ticket like this one is... closed...

violine1101

MC-138550 was closed because it contained literally every issue ever with performance and was not specific in any way, shape or form. The developers did not know how to handle it, so it was closed.

If you're not willing to help with resolving the issue, then why complain here? This is a community bug tracker, managed by us volunteers, which happens to be used and hosted by Mojang. Yes, if Mojang wants to search for the causes by themselves, they of course can do that, but don't be surprised if it takes ages for the performance improvements to take effect.

As a matter of fact, I haven't seen a single debug report since the release of 1.14.4 that shows any bad server performance here on the bug tracker.

migrated

About debug reports - maybe this is because after 1.14.4 you have closed the most important ticket about performance and stated that 1.14.4 is final 1.14 release and there will be no more performance fixes and you now focus on releasing 1.15 in which you eventually might start working on performance again. So people just gave up.

migrated
[media]

I also have the same problems and I can even show it using an image as I host both 1.12.2 servers and one 1.14.4 server, and if you look at the image that I have attached, it is the upper one being the 1.14.4 server with 1 player on, and the lower one a 1.12.2 server with 1 player on.
I have chosen to strip all other things as they are not necessary to illustrate it.

 

I'am sending the debug report in the next message, so only the Users can see it.

 

The server have:

2x Intel Xeon E5603
Minecraft Server have 8 GB RAM
The operating system is Windows Server 2016 64bit
Java Version: 1.8.0_201 64bit

 

Sometimes the Minecraft server with 1.14.4 stops itself and reports that it is doing so because it is running too many ticks behind.

 

I get this message right after I restart the server, even if no players are joined.

[16:31:37] [Server thread/WARN]: Can't keep up! Is the server overloaded? Running 2364ms or 47 ticks behind

 

CubeTheThird

This ticket seems to have devolved into a thread for complaints, rather than being constructive. As this is a bug tracker and not a discussion forum, we ask that further conversation about this ticket, performance, and other topics be taken to the Mojira Subreddit.

Any further comments here should be constructive to the resolution of this problem, including providing debug reports and profiler logs, steps to recreate the issue, or other helpful insights on the problem at hand.

As the creator of this ticket has stated though, the scope of this report is becoming broad, and may need to be split up to address specific performance issues. Discussions around that should also be left to the subreddit.

migrated

https://bugs.mojang.com/browse/MC-160373 My bug report contains debug 🙂

slicedlime

Thank you for the report and all the information in this ticket!

However, we are now closing this, as it is too broad and vague to be possible for us to act on. We are continuously working on performance improvements, and if you have information about specific features or scenarios in the game that cause performance degradation, please post new, specific bug reports for those cases. Make sure to include:

  • Reproduction steps

  • Screenshots with the debug performance graph open

  • Profiler data (file written by doing /debug start followed by /debug stop) captured when performance was poor

  • Debug reports (file written by doing /debug report)

If the problem is not easily reproducible in a freshly created world, it would also be helpful if you provide your world as a shared download.

Again, thank you for all the help in tracking down these issues!

migrated

Hi i have same issue. I have big tps loss with 20 and more players.

I think it's because the server does not use all of the server threads. It always performs an operation on one thread and, when the operation is complete, starts another operation on the next thread.

Debug Log

Server version 1.15.2

Hosting server specs:
name: Linux
version: 4.19.0-6-amd64
jvmversion: 1.8.0_222
arch: amd64
maxmem: 14316732416
cpu: 16
runtime: 36848742
flags: -Xms1G -Xmx15G

wobst.michael

@unknown your attached debug report is useless in this case, as it's from a modded server (paper). We only support unmodified vanilla Minecraft.

migrated

Sorry i uploaded bad file for debug. There is new

[media]

Yeah. I know but the idea is the same server uses in one moment only on one thread.

migrated

(Unassigned)

Confirmed

Performance

cpu, lag, ram, server, server-side, tps, usage

Minecraft 1.12 Pre-Release 7, Minecraft 1.12, Minecraft 1.12.2, Minecraft 17w45b, Minecraft 17w46a, ..., 19w39a, 19w40a, 19w41a, 19w42a, 19w45b

Retrieved