Ahhh... That would be why my items were not destroyed in my "trash can" earlier today. I had to change the cactus out for lava.
I figured out why the "bug" is happening. It is because I got the map from the cartographer, in the nether (that is where my trading hall is.)
It is obvious what is going on here... The mobs are walking out of "loaded chunks" and are not de-spawning. Therefore they build up in massive numbers.