Fix map container mutation issues #5103

Job	Run time
Mac (Debug, clang, x86_64)	6m 40s
Windows (Debug, MSVC, Win32)	16m 38s
Mac (Debug, clang, arm64)	8m 18s
Linux (Debug, gcc-9)	9m 47s
Windows (Debug, MSVC, x64)	15m 40s
Mac (Release, clang, x86_64)	7m 44s
Linux (Debug, gcc-13)	2m 16s
Windows (Release, MSVC, Win32)	20m 32s
Mac (Release, clang, arm64)	6m 59s
Linux (Debug, clang-16)	9m 14s
Windows (Release, MSVC, x64)	20m 25s
Linux (Release, gcc-9)	12m 28s
Linux (Release, gcc-13)	11m 18s
Linux (Release, clang-16)	15m 5s
Linux (Debug, gcc-13, -DFSO_BUILD_WITH_OPENGL=OFF -DFSO_BUILD_WITH_VULKAN=OFF)	8m 4s
Linux (Release, gcc-13, -DFSO_BUILD_WITH_OPENGL=OFF -DFSO_BUILD_WITH_VULKAN=OFF)	6m 43s
	2h 57m 51s

Provide feedback