mojira.dev
MC-128163

Worldgen worker thread crash - Out of Memory error

When generating terrain, the Worldgen worker threads crash with Out of memory errors. This is especially likely to happen while generating Jungle biomes.

The crashing of the worker threads is severe. When it happens, the game stops responding to attempts to save the game. Pressing Save and Exit causes the game to hang without doing anything and the game must be terminated externally (eg with Task Manager).

Some examples of crashes from the log:

Caught exception in thread Thread[WorldGen-Worker-93,5,main]
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ForkJoinPool.tryAddWorker(ForkJoinPool.java:1338)
	at java.util.concurrent.ForkJoinPool.deregisterWorker(ForkJoinPool.java:1460)
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:167)
	
Caught exception in thread Thread[WorldGen-Worker-46,5,main]
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ForkJoinPool.tryAddWorker(ForkJoinPool.java:1338)
	at java.util.concurrent.ForkJoinPool.deregisterWorker(ForkJoinPool.java:1460)
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:167)

Caught exception in thread Thread[WorldGen-Worker-29,5,main]
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ForkJoinPool.tryAddWorker(ForkJoinPool.java:1338)
	at java.util.concurrent.ForkJoinPool.deregisterWorker(ForkJoinPool.java:1460)
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:167)

Related issues

Attachments

Comments

migrated
[media]
bdm68

This seemed to happen because I was pushing the JVM rather hard with experimental options at the same time that I was stress testing the terrain generation by running it as fast as possible for up to an hour. This is likely the cause of the failure. I have yet to reproduce this with more sensible JVM options (-Xmx1024M and nothing else).

The JVM parameters: -Xms1024m -Xmx1024m -XX:+UnlockExperimentalVMOptions -XX:+UseG1GC -XX:G1NewSizePercent=20 -XX:G1ReservePercent=20 -XX:MaxGCPauseMillis=50 -XX:G1HeapRegionSize=16M

bdm68

It has happened again in 18w15a with vanilla JVM settings (-Xmx1024M):

Caught exception in thread Thread[WorldGen-Worker-295,5,main]
java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:714)
at java.util.concurrent.ForkJoinPool.tryAddWorker(ForkJoinPool.java:1338)
at java.util.concurrent.ForkJoinPool.deregisterWorker(ForkJoinPool.java:1460)
at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:167)

kumasasa

Is this still an issue in the current snapshot 18w19b ?

bdm68

I have been unable to run 18w19b and 18w20a because it crashes on startup (MC-129374).

bdm68

Still waiting for MC-129374 to be fixed before I can test this.

bdm68

MC-129374 hasn't been fixed yet, but I have been able to get 18w20c working by increasing the stack size.

The issue is still present in 18w20c. I am not getting the WorldGen-Worker error messages in the logs, but the other symptoms are still present: new chunks stop loading and the game can no longer be saved. Environment is 32-bit Java.

To reproduce, try running with limited memory (1Gb) and simply create new terrain by flying in a straight line in creative mode. When the memory reaches its limit, the issue is likely to happen.

bdm68

After some more investigation, I suspect there is a memory leak with world generation. The memory in use keeps climbing as more terrain is generated, even after major GC cycles that occur every few minutes. On world load on 32-bit Java with 1G memory, the memory in use initially has a range of 25%-45%. This slowly creeps up after each GC cycle, sometimes dropping with a major GC cycle until it reaches somewhere around 75%-95% or so, at which point a major GC cycle can no longer recover the memory in use. Then the Worldgen threads crash and saving the game is no longer possible.

I have also discovered a workaround. Save the game periodically and then reload it. This can recover up to 500M of lost memory.

bdm68

Confirmed for 18w21a.

bdm68

Confirmed for 18w20c using 64-bit Java on Windows 10.

Cannot reproduce for 18w22a using 64-bit Java on Windows 10. May be fixed. Need independent confirmation in 32-bit environment, preferably on Windows 7. I no longer have reliable access to 32-bit Java on Windows 7 due to replacing a defective hard disk.

 

kumasasa

Assuming fixed since there is no difference between 64bit and 32bit JVM when having 1GB heap available (difference is stack, see e.g. MC-129374, but this is a different story)

bdm68

I concur with the "assume fixed" assessment. I tried really hard to crash it in 18w22a and I could not do so even though I flew over 25,000 blocks. In 18w20c I crashed it after 3,000 blocks. The save I generated in18w20c is linked in MC-130162.

bdm68

Affects 1.13 pre-1.

Generating chunks by flying around with commands will occasionally cause the world generation to stop and saving to become impossible. Reproduced on Win10 using 64-bit Java. It is difficult to reproduce but not impossible.

kumasasa

@unknown, after having this issue, please attach the crash report and the launcher log.

bdm68
[media]

attached after experiencing the issue.

I'm not sure if this is still the same issue or if it's something else.

bdm68

This core issue is resolved. I am experiencing other issues, in particular MC-129492.

bdm68

(Unassigned)

Unconfirmed

Minecraft 18w14b, Minecraft 18w15a, Minecraft 18w20c, Minecraft 18w21a

Minecraft 18w22c

Retrieved