Support MIPS64 - write correct PC register width on uc_emu_start

[OBarronCS]

This PR allows MIPS64 to work - previously MIPS64 would crash immediately after starting the emulator - see #2109.

The current uc_emu_start code makes the assumptions that MIPS has a 32-bit program counter. Adding a check to see if it's 64-bit and writing the correct value to PC makes MIPS64 run.

unicorn/uc.c

Lines 1021 to 1026 in 8d52ece

	#ifdef UNICORN_HAS_MIPS
	case UC_ARCH_MIPS:
	// TODO: MIPS32/MIPS64/BIGENDIAN etc
	uc_reg_write(uc, UC_MIPS_REG_PC, &begin_pc32);
	break;
	#endif

[amaanq]

I'm not an expert of unicorn, but, I'd think this should be UC_MODE_MIPS64, no? And based on the TODO, it was just never accounted for...

See here for an example

[OBarronCS]

I suppose UC_MODE_MIPS64 is more correct and forward-compatible. Right now, it is the same value as UC_MODE_64 (the constant value 8).

[OBarronCS]

I see - I assumed UC_MODE_64 was a generic 64-bit mode flag, but on further look it is only meant to apply for x86 based on the code and just so happens to line up with the MIPS64 flag

unicorn/include/unicorn/unicorn.h

Lines 139 to 142 in 8d52ece

	// x86 / x64
	UC_MODE_16 = 1 << 1, // 16-bit mode
	UC_MODE_32 = 1 << 2, // 32-bit mode
	UC_MODE_64 = 1 << 3, // 64-bit mode

I made this change.

[wtdcode]

Thanks for giving a try. Looks good to me and will merge once CI passed.

[OBarronCS]

I found an edge case related to MIPS64 TLB that needs to be considered. Posting an update in a few.

[wtdcode]

I found an edge case related to MIPS64 TLB that needs to be considered. Posting an update in a few.

Please also consider adding a (maybe simple) test case so that we won't break it in our future release.

[OBarronCS]

There seems to be limited functionality even with UC_TLB_VIRTUAL enabled in MIPS64. Some instructions work, others do not. Unfortunately, there seems to be more at play here. I'm sometimes getting exception #20 on seemingly random instructions (one's that don't concern memory).

#!/usr/bin/env python3
from unicorn import *
from unicorn.mips_const import *

print(unicorn.uc_version())

emu = Uc(UC_ARCH_MIPS, UC_MODE_64 | UC_MODE_BIG_ENDIAN)
emu.ctl_set_tlb_mode(UC_TLB_VIRTUAL)
# def hook(uc, vaddr, a, b, c):
#     print(hex(vaddr), a, b, c)
#     return True
# emu.hook_add(UC_HOOK_TLB_FILL, hook)

# daddu  $gp, $gp, $ra
code = b"\x03\x9f\xe0\x2d" * 2

ADDRESS =  0x1000
ADDRESS = 0x555555d56370 & ~0xFFF

emu.mem_map(ADDRESS, 4 * 1024)
emu.mem_write(ADDRESS, code)

# emu.reg_write(UC_MIPS_REG_CP0_STATUS, 0b000)
emu.reg_write(UC_MIPS_REG_PC, ADDRESS)
emu.reg_write(UC_MIPS_REG_GP, 0x1)
emu.reg_write(UC_MIPS_REG_RA, 0x1)

try:
    emu.emu_start(ADDRESS, 0, count=2)
except Exception as e:
    print(e)

print(f"PC: {hex(emu.reg_read(UC_MIPS_REG_PC))}")

Additionally, for future people who might try to get this to work, here are other things I tried:

I tried to get this to work without UC_TLB_VIRTUAL, however addresses over 0x80000000 (the 32-bit usermode limit) will still give an exception, and I was unable to find the root cause of this.

Basically, in 64-bit mode, the TLB has the following translation scheme:
(page 29 in this doc - https://s3-eu-west-1.amazonaws.com/downloads-mips/documents/MD00091-2B-MIPS64PRA-AFP-05.04.pdf)

You can find qemu doing these translations/memory checks here https://github.com/qemu/qemu/blob/495de0fd82d8bb2d7035f82d9869cfeb48de2f9e/target/mips/system/physaddr.c#L145

I assumed that the following location was doing a final step of the mapping logic, but I was incorrect.

unicorn/qemu/target/mips/helper.c

Lines 48 to 53 in f2e80ff

	int fixed_mmu_map_address(CPUMIPSState *env, hwaddr *physical, int *prot,
	target_ulong address, int rw, int access_type)
	{
	if (address <= (int32_t)0x7FFFFFFFUL) {
	if (!(env->CP0_Status & (1 << CP0St_ERL))) {
	*physical = address + 0x40000000UL;

[wtdcode]

For accessing high address, you need to switch to supervised levels? Not an MIPS expert unfortunately =(

[OBarronCS]

I played around with bits of the status register, and didn't see any difference in behavior. It appears to throw exceptions on certain types of instructions, even ones that do not read/write from memory

Here is the output of the Unicorn/qemu logging with Python script in my previous comment, running the single instruction daddu $gp, $gp, $ra which adds two registers.

*** TCG before optimization:
 0:  ld_i32 tmp0,env,$0xfffffffffffffff0
 1:  movi_i32 tmp1,$0x0
 2:  brcond_i32 tmp0,tmp1,lt,$L0

 insn_idx=0 ---- 0000000000001000 0000000000000000 0000000000000000
 1:  movi_i64 PC,$0x1000
 2:  movi_i32 tmp0,$0x4
 3:  movi_i64 tmp2,$0x5d79b1a4b810
 4:  movi_i64 tmp3,$0x1000
 5:  movi_i64 tmp4,$0x0
 6:  call hookcode_4_7c55f8fbd01e,$0x0,$0,tmp2,tmp3,tmp0,tmp4
 7:  ld_i32 tmp0,env,$0xfffffffffffffff0
 8:  movi_i32 tmp1,$0x0
 9:  brcond_i32 tmp0,tmp1,lt,$L0
 10:  movi_i32 tmp0,$0x14
 11:  movi_i32 tmp1,$0x0
 12:  movi_i64 PC,$0x1000
 13:  call raise_exception_err,$0x8,$0,env,tmp0,tmp1
 14:  movi_i32 tmp0,$0x14
 15:  movi_i32 tmp1,$0x0
 16:  call raise_exception_err,$0x8,$0,env,tmp0,tmp1
 17:  add_i64 gp,gp,ra
 18:  set_label $L0
 19:  exit_tb $0x7c55b8a00043

*** TCG before codegen:
 0:  ld_i32 tmp0,env,$0xfffffffffffffff0  dead: 1
 1:  movi_i32 tmp1,$0x0
 2:  brcond_i32 tmp0,tmp1,lt,$L0  dead: 0 1

 insn_idx=0 ---- 0000000000001000 0000000000000000 0000000000000000
 1:  movi_i64 PC,$0x1000  sync: 0  dead: 0
 2:  movi_i32 tmp0,$0x4
 3:  movi_i64 tmp2,$0x5d79b1a4b810
 4:  movi_i64 tmp3,$0x1000
 5:  movi_i64 tmp4,$0x0
 6:  call hookcode_4_7c55f8fbd01e,$0x0,$0,tmp2,tmp3,tmp0,tmp4  dead: 0 1 2 3
 7:  ld_i32 tmp0,env,$0xfffffffffffffff0
 8:  movi_i32 tmp1,$0x0
 9:  brcond_i32 tmp0,tmp1,lt,$L0  dead: 0 1
 10:  movi_i32 tmp0,$0x14
 11:  movi_i32 tmp1,$0x0
 12:  movi_i64 PC,$0x1000  sync: 0  dead: 0
 13:  call raise_exception_err,$0x8,$0,env,tmp0,tmp1  dead: 0 1 2
 14:  set_label $L0
 15:  exit_tb $0x7c55b8a00043

# This raises an exception!

*** TCG before optimization:
 0:  ld_i32 tmp0,env,$0xfffffffffffffff0
 1:  movi_i32 tmp1,$0x0
 2:  brcond_i32 tmp0,tmp1,lt,$L0

 insn_idx=0 ---- 0000000000000000 0000000000000000 0000000000000000
 1:  call wait,$0x0,$0,env
 2:  set_label $L0
 3:  exit_tb $0x7c55b8a00183

*** TCG before codegen:
 0:  ld_i32 tmp0,env,$0xfffffffffffffff0
 1:  movi_i32 tmp1,$0x0
 2:  brcond_i32 tmp0,tmp1,lt,$L0  dead: 0 1

 insn_idx=0 ---- 0000000000000000 0000000000000000 0000000000000000
 1:  call wait,$0x0,$0,env  dead: 0
 2:  set_label $L0
 3:  exit_tb $0x7c55b8a00183

[wtdcode]

call raise_exception_err

So the execption was thrown during translation. I will have a look shortly.

[OBarronCS]

I found the bug that was preventing certain instructions from being executed - the CPU model was still being set to a 32-bit MIPS model, instead of a 64-bit one.

With the latest commit, and with UC_TLB_VIRTUAL enabled, MIPS64 appears to be working great now! I'll make another commit to add a test to Unicorn.

This setup now works great. If you use MIPS64, make sure to enable UC_TLB_VIRTUAL. Otherwise, accesses at high addresses will fail. I put some notes on this below, in case someone in the future tries to tackle this issue:

Notes on further fixing MIPS64 memory lookups

Without TLB_VIRTUAL, there is some more work needed to get memory accesses functioning for addresses > 0x7FFFFFFFUL . I briefly looked into it where it is currently crashing - here are some notes on it if anyone in the future tries to take this up:

• Without UC_TLB_VIRTUAL enabled, memory accesses will crash on addresses > 0x7FFFFFFFUL because it takes the following codepath in the memory access:
  

  unicorn/qemu/target/mips/helper.c

  Lines 266 to 274 in 0f45f15

  	#if defined(TARGET_MIPS64)
  	} else if (address < 0x4000000000000000ULL) {
  	/* xuseg */
  	if (UX && address <= (0x3FFFFFFFFFFFFFFFULL & env->SEGMask)) {
  	ret = env->tlb->map_address(env, physical, prot,
  	real_address, rw, access_type);
  	} else {
  	ret = TLBRET_BADADDR;
  	}

(see earlier comment on MIPS64 mappings #2111 (comment))

The UX variable comes from the MIPS status flag which indicates "user mode", which must be explicitly set by the user (it is not enabled by default).

# The `1` at the 5th bit position indicates access to 64-bit User Segments (which go from address 0 to 0x3FFF_FFFF_FFFF_FFFF
status_register = 0b0100_0000_0000_0000_0010_0100
# I found the other `1` bits in this string to be necessary as well
emu.reg_write(UC_MIPS_REG_CP0_STATUS, status_register)

Reference - "MIPS Architecture For Programmers Volume III: MIPS64..."

• https://s3-eu-west-1.amazonaws.com/downloads-mips/documents/MD00091-2B-MIPS64PRA-AFP-06.03.pdf
• Section 9.33 Status Register (page 213 in the PDF)

The call to env->tlb->map_address is a function pointer to the r4k_map_address function. Currently, the loop in that function will run a couple times and then eventually return TLBRET_NOMATCH. Looks like it might be necessary to setup some structure/registers for the mapping logic.

[wtdcode]

I would rather to index the cpu like UC_CPU_MIPS32_ENDING + uc->cpu_model instead if chaning the enum value here.

[OBarronCS]

I made this change 👍

[wtdcode]

Thanks for the valuable analysis. I was busy with distributions issues and didn't have a look yet sorry. For the VTLB issue, it is definitely a bug and I will have a look.

[OBarronCS]

The CI is failing with a strange error:

  File "/home/runner/work/unicorn/unicorn/tests/regress/mips64.py", line 17, in runTest
    uc.ctl_set_tlb_mode(UC_TLB_VIRTUAL)
AttributeError: 'Uc' object has no attribute 'ctl_set_tlb_mode'

The Python package in the CI/CD is not having the ctl_set_tlb_mode?

[Antelox]

Since we run tests for the Python 2.7 wheel too, we need to add a decorator to skip the test when the test itself is using a feature not available for the Python 2.

@unittest.skipIf(sys.version_info < (3, 7), reason="requires python3.7 or higher")

[OBarronCS]

Good call - done

[wtdcode]

@OBarronCS I'm going to merge this, thanks!

Do you have a PoC for the VTLB bug or would you like to raise another standalone issue for that?

[wtdcode]

Thanks! Also, please raise another issue about the virtual TLB at your convenience.

[OBarronCS]

Great! I will make an issue related to the MIPS64 memory mappings so that it's searchable for future contributors.