Minecraft 1.17 using abnormally high amounts of RAM (far above what it's allocated)

After a couple minutes, my Minecraft 1.17.1 game begins using 8-12 GB of RAM. It is only allocated 4 G. This happens in both vanilla and Fabric Mod Loader. It sometimes crashes my pc as well. Any ideas as to how I can fix this?