Verifying correctness of code implementations

[xezon]

The overall game code complexity is quite large. This is problematic for keeping error counts in check when reimplementing original game functions and classes. I wonder how feasible it would be to automate this a bit. Assuming we would compile Thyme with the same compiler and compile flags EA Games used for it, then we should get the same assembler output, minus the static addresses. Maybe we could have a system that compares Thyme and hooked function assembly on initialization and assert if the function contents don't match. This way it would be easier to catch deviances. Ideally such functionality can be toggled on and off for individual functions. And ideally it is used on as many functions as possible to validate correctness. Such approach would strengthen confidence in correctness of Thyme. For best results, all custom Thyme code in game functions would have to be compiled out.

[xezon]

tomsons26

binary matching would be ideal yes

Target machine: x32
@comp.id   id version count   description
000b225e    b   8798     3
00132359   13   9049     2
000c1c7b    c   7291     7
000b1f6f    b   8047     3
000e1c83    e   7299     6
000a1f6f    a   8047    11
00041f6f    4   8047     2
00041fe8    4   8168     2 [LNK] VS98 (6.0) imp/exp build 8168
00302354   30   9044     1
001220fc   12   8444     1
00000000    0      0    10 [---] Unmarked objects (old)
00131f56   13   8022    12
00131f62   13   8034    20
001c23da   1c   9178     2
00010000    1      0   398 [---] Unmarked objects
001923fa   19   9210     2
001d23da   1d   9178    81
000a2636    a   9782    67 [ C ] VS98 (6.0) SP6 build 8804
000b2636    b   9782   866 [C++] VS98 (6.0) SP6 build 8804
000606c7    6   1735     1 [RES] VS98 (6.0) SP6 cvtres build 1736
000420ff    4   8447    77 [LNK] VC++ 6.0 SP5 imp/exp build 8447

is the rich header info from 1.04 exe
so VS6 SP6 was used, linker im not sure whats that about, don't think you can just use SP5 linker
for doing diffs there is
https://www.zynamics.com/bindiff.html
https://www.zynamics.com/software.html
https://github.com/joxeankoret/diaphora

ideally i guess would be making something that diffs at assembly level but ignoring addresses and other changing variables
N64 re'ers have written such a thing for MIPS
https://github.com/simonlindholm/asm-differ

[xezon]

Duncanspumpkin

I think binary matching is a bad idea. It just leads to far too slow development. I tried to work on an n64 project but there's just no reason to try 500 slight modifications when the code is already correct
The best thing to do is to get to a standalone project as quickly as possible and from there you can start removing all the custom containers and use just normal standard containers which are in general much more bug free.
Yes the occasional bug will be introduced but then if you get a standalone implementation going then you will have way more people testing and trying out the game so you can squash those bugs.
For OpenRCT2 we have a bunch of Replay tests that run against every single pull request that ensure that game state isn't accidentally modified. N64 re projects also have their TAS replays they use to check changes. I think something like that would be a good idea but I don't know enough about the engine to know if it's possible/easily achievable at this time.

[xezon]

In my opinion, best case assembler compare tool, for Thyme, would work like this:

1. Tool locates functions in original and Thyme executable files, can use pattern matching theFunctionName, The_Function_Name to solve name differences *1
2. Tool compares x86 assembler instructions and takes notes of differences
3. Tool tries to link faulty assembler instructions back to Thyme source code *2
4. Tool gives status report of passed and failed function checks. Lists assembler and source code differences under failed functions.
5. Tool gives option to provide a function whitelist, to exclude any functions that may have custom changes but have been verified to be a match to original otherwise

With such tool, it should be straight forward to crawl through the assembler diffs and tick them off, even if there are already some custom code changes in Thyme.

*1 Function to assembler perhaps is searchable with dbghelp api
*2 Should be doable with dbghelp api

[xezon]

I expect the most critical parts are:

1. Properly locating the matching function begin and end in both Thyme DLL and game.dat
2. Properly matching the Thyme DLL assembler with the Thyme source code
3. Voiding/Normalizing all addresses and offsets in both Thyme DLL and game.dat that naturally would be different *1
4. Functionality to walk x86 assembler from one instruction to another

*1 For example, a jmp to fixed address would certainly be different.

Once we have these core mechanisms, the rest should be straight forward to build around.

1 Properly locate matching function begin and end in both Thyme DLL and game.dat

For game.dat we either need a function pdb or custom function to address lookup table. This needs to be just generated once and is expected to be correct and complete.

For Thyme we simply need the EXE or DLL and its pdb.

We can enumerate symbols with dbghelp's SymEnumSymbols function:
https://learn.microsoft.com/en-us/windows/win32/api/dbghelp/nf-dbghelp-symenumsymbols
https://learn.microsoft.com/en-us/windows/win32/api/dbghelp/ns-dbghelp-symbol_info
https://learn.microsoft.com/en-us/windows/win32/debug/enumerating-symbols

2 Properly match the Thyme DLL assembler with the Thyme source code

We can locate source files and line of address with dbghelp's SymGetLineFromAddr function:
https://learn.microsoft.com/en-us/windows/win32/api/dbghelp/nf-dbghelp-symgetlinefromaddr
https://learn.microsoft.com/en-us/windows/win32/api/dbghelp/ns-dbghelp-imagehlp_line

It also possible to walk adjacent source lines with SymGetLineNext, SymGetLinePrev:
https://learn.microsoft.com/en-us/windows/win32/api/dbghelp/nf-dbghelp-symgetlinenext
https://learn.microsoft.com/en-us/windows/win32/api/dbghelp/nf-dbghelp-symgetlineprev

3 Void/Normalize all addresses and offsets in both Thyme DLL and game.dat

We need to make sure that all relative addresses are normalized to same bases, or void them for comparisons.

According to tomsons26, point 3 is covered with
https://github.com/OmniBlade/unassemblize

4 Functionality to walk x86 assembler instructions

There is x86 reference.
http://ref.x86asm.net/coder32.html

To walk assembler instructions, we need x86 assembler parser, that has knowledge of opcodes and their respective operand counts and operand sizes. When this info is available, we can walk from one instruction to another and compare one by one and get accurate records of which instruction(s) differ.

tomsons26 has linked to following projects achieving that:
https://github.com/simonlindholm/asm-differ
https://github.com/encounter/objdiff

I expect that we can take disasm code from ollydbg project as a basis. It has full knowledge of x86 assembler.
https://github.com/x64dbg/OllyDbg/tree/master/OllyDbg