[DNM] Try to fix CI test for qemu xtensa mmu #682

Job	Run time
Setup	7s
Host Tools (linux-x86_64)	33m 6s
Toolchain aarch64-zephyr-elf (linux-x86_64)	25m 58s
CMake Package (linux-x86_64)	37s
Toolchain arc64-zephyr-elf (linux-x86_64)	32m 6s
Toolchain arc-zephyr-elf (linux-x86_64)	1h 20m 7s
Toolchain arm-zephyr-eabi (linux-x86_64)	1h 35m 20s
Toolchain microblazeel-zephyr-elf (linux-x86_64)	35m 5s
Toolchain mips-zephyr-elf (linux-x86_64)	33m 6s
Toolchain nios2-zephyr-elf (linux-x86_64)	37m 22s
Toolchain riscv64-zephyr-elf (linux-x86_64)	1h 7m 30s
Toolchain sparc-zephyr-elf (linux-x86_64)	52m 26s
Toolchain x86_64-zephyr-elf (linux-x86_64)	29m 49s
Toolchain xtensa-dc233c_zephyr-elf (linux-x86_64)	22m 6s
Toolchain xtensa-espressif_esp32_zephyr-elf (linux-x86_64)	22m 22s
Toolchain xtensa-espressif_esp32s2_zephyr-elf (linux-x86_64)	21m 32s
Toolchain xtensa-espressif_esp32s3_zephyr-elf (linux-x86_64)	20m 54s
Toolchain xtensa-intel_ace15_mtpm_zephyr-elf (linux-x86_64)	20m 48s
Toolchain xtensa-intel_tgl_adsp_zephyr-elf (linux-x86_64)	22m 7s
Toolchain xtensa-mtk_mt8195_adsp_zephyr-elf (linux-x86_64)	20m 47s
Toolchain xtensa-nxp_imx_adsp_zephyr-elf (linux-x86_64)	21m 53s
Toolchain xtensa-nxp_imx8m_adsp_zephyr-elf (linux-x86_64)	22m 8s
Toolchain xtensa-nxp_imx8ulp_adsp_zephyr-elf (linux-x86_64)	20m 47s
Toolchain xtensa-nxp_rt500_adsp_zephyr-elf (linux-x86_64)	20m 22s
Toolchain xtensa-sample_controller_zephyr-elf (linux-x86_64)	20m 30s
Distribution Bundle (linux-x86_64)	14m 47s
Test (ubuntu-20.04-x86_64) Subset 1	7m 32s
Test (ubuntu-20.04-x86_64) Subset 2	8m 29s
Test Result	0s
	13h 29m 43s

Provide feedback