[MC-128163] Worldgen worker thread crash

When generating terrain, the Worldgen worker threads crash with Out of memory errors. This is especially likely to happen while generating Jungle biomes.

The crashing of the worker threads is severe. When it happens, the game stops responding to attempts to save the game. Pressing Save and Exit causes the game to hang without doing anything and the game must be terminated externally (eg with Task Manager).

Some examples of crashes from the log:

Caught exception in thread Thread[WorldGen-Worker-93,5,main]
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ForkJoinPool.tryAddWorker(ForkJoinPool.java:1338)
	at java.util.concurrent.ForkJoinPool.deregisterWorker(ForkJoinPool.java:1460)
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:167)
	
Caught exception in thread Thread[WorldGen-Worker-46,5,main]
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ForkJoinPool.tryAddWorker(ForkJoinPool.java:1338)
	at java.util.concurrent.ForkJoinPool.deregisterWorker(ForkJoinPool.java:1460)
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:167)

Caught exception in thread Thread[WorldGen-Worker-29,5,main]
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at java.util.concurrent.ForkJoinPool.tryAddWorker(ForkJoinPool.java:1338)
	at java.util.concurrent.ForkJoinPool.deregisterWorker(ForkJoinPool.java:1460)
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:167)

Linked issues

relates to 2

MC-129492 Moving through world gradually uses up all memory, causes stuttering and crash Resolved

MC-130162 Exception generating new chunk - OutOfMemoryError: Java heap space (32 bit Java) Resolved

Attachments

crash-2018-06-16_12.41.30-server.txt

Comments 15

bdm68 2018-04-12T07:35:30Z

This seemed to happen because I was pushing the JVM rather hard with experimental options at the same time that I was stress testing the terrain generation by running it as fast as possible for up to an hour. This is likely the cause of the failure. I have yet to reproduce this with more sensible JVM options (-Xmx1024M and nothing else).

The JVM parameters: -Xms1024m -Xmx1024m -XX:+UnlockExperimentalVMOptions -XX:+UseG1GC -XX:G1NewSizePercent=20 -XX:G1ReservePercent=20 -XX:MaxGCPauseMillis=50 -XX:G1HeapRegionSize=16M

bdm68 2018-04-12T12:12:32Z

It has happened again in 18w15a with vanilla JVM settings (-Xmx1024M):

Caught exception in thread Thread[WorldGen-Worker-295,5,main]
java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:714)
at java.util.concurrent.ForkJoinPool.tryAddWorker(ForkJoinPool.java:1338)
at java.util.concurrent.ForkJoinPool.deregisterWorker(ForkJoinPool.java:1460)
at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:167)

kumasasa 2018-05-09T18:53:15Z

Is this still an issue in the current snapshot 18w19b ?

bdm68 2018-05-13T13:10:45Z

I have been unable to run 18w19b and 18w20a because it crashes on startup (MC-129374).

bdm68 2018-05-16T23:14:30Z

Still waiting for MC-129374 to be fixed before I can test this.

5 more comments

bdm68 2018-05-18T00:41:59Z

MC-129374 hasn't been fixed yet, but I have been able to get 18w20c working by increasing the stack size.

The issue is still present in 18w20c. I am not getting the WorldGen-Worker error messages in the logs, but the other symptoms are still present: new chunks stop loading and the game can no longer be saved. Environment is 32-bit Java.

To reproduce, try running with limited memory (1Gb) and simply create new terrain by flying in a straight line in creative mode. When the memory reaches its limit, the issue is likely to happen.

bdm68 2018-05-18T03:34:20Z

After some more investigation, I suspect there is a memory leak with world generation. The memory in use keeps climbing as more terrain is generated, even after major GC cycles that occur every few minutes. On world load on 32-bit Java with 1G memory, the memory in use initially has a range of 25%-45%. This slowly creeps up after each GC cycle, sometimes dropping with a major GC cycle until it reaches somewhere around 75%-95% or so, at which point a major GC cycle can no longer recover the memory in use. Then the Worldgen threads crash and saving the game is no longer possible.

I have also discovered a workaround. Save the game periodically and then reload it. This can recover up to 500M of lost memory.

bdm68 2018-05-24T00:13:21Z

Confirmed for 18w21a.

bdm68 2018-05-30T08:32:49Z

Confirmed for 18w20c using 64-bit Java on Windows 10.

Cannot reproduce for 18w22a using 64-bit Java on Windows 10. May be fixed. Need independent confirmation in 32-bit environment, preferably on Windows 7. I no longer have reliable access to 32-bit Java on Windows 7 due to replacing a defective hard disk.

kumasasa 2018-06-01T10:53:05Z

Assuming fixed since there is no difference between 64bit and 32bit JVM when having 1GB heap available (difference is stack, see e.g. MC-129374, but this is a different story)

bdm68 2018-06-06T05:56:05Z

I concur with the "assume fixed" assessment. I tried really hard to crash it in 18w22a and I could not do so even though I flew over 25,000 blocks. In 18w20c I crashed it after 3,000 blocks. The save I generated in18w20c is linked in MC-130162.

bdm68 2018-06-12T08:44:45Z

Affects 1.13 pre-1.

Generating chunks by flying around with commands will occasionally cause the world generation to stop and saving to become impossible. Reproduced on Win10 using 64-bit Java. It is difficult to reproduce but not impossible.

kumasasa 2018-06-12T18:53:44Z

@unknown, after having this issue, please attach the crash report and the launcher log.

bdm68 2018-06-16T02:49:00Z

[media]

attached after experiencing the issue.

I'm not sure if this is still the same issue or if it's something else.

bdm68 2018-06-16T02:52:57Z

This core issue is resolved. I am experiencing other issues, in particular MC-129492.

Worldgen worker thread crash - Out of Memory error

Linked issues

relates to 2

Attachments

Comments 15