First round of F3DEX3 changes for basic features #258

sauraen · 2023-10-15T00:25:31Z

Most of these changes were made months ago, but finally now re-synced with current fast64.

Additions for F3DEX3:

GBI modifications and new command encodings
Packed normals export / import (for simultaneous vertex colors + lighting)
Export / import of basic new commands: SPAmbOcclusion, SPFresnel, SPAttrOffsetST, SPAttrOffsetZ

Changes affecting non-F3DEX3 fast64:

Revamped geometry mode bits UI. Bits are grouped and have "smart" warning / info messages which detect probably wrong combinations of settings. This is especially needed for F3DEX3 because the options interact in nontrivial ways.
Removed passing microcode type and hardware V1 flag through many functions and classes. This was previously needed to create F3D instances, but doing so was already deprecated in favor of get_F3D_GBI. The new code is simpler and should be (slightly) faster.

What is NOT included in this PR:

Any changes to nodes. The new F3DEX3 features (e.g. simultaneous vertex colors + lighting) will not be shown in the preview. They are still exported and imported correctly though.
New F3DEX3-only presets. This will come when the new cel shading PR gets made, as I already have a system for this there.

…EX3 changes

Yanis42

few comments here and there, OoT exports properly after fixing the OOTModel call from oot_level_classes.py, the import is broken though

most of these comments are either missing type hints where I saw them (probably more?) and few things f-strings can do better imo, the code itself looks good otherwise

fast64_internal/utility.py

fast64_internal/f3d/f3d_gbi.py

Yanis42 · 2023-10-16T17:45:56Z

fast64_internal/f3d/f3d_material.py

@@ -3820,7 +4084,15 @@ def mat_register():
        items=enumF3D,
        default="F3D",
    )
-    bpy.types.Scene.isHWv1 = bpy.props.BoolProperty(name="Is Hardware v1?")
+    bpy.types.Scene.isHWv1 = bpy.props.BoolProperty(


is this really needed? I thought hardware v1 was really really uncommon, I'm just wondering if we should keep support for it

There's only a couple SGI Indy N64 devkits owned by collectors which have V1 chips. (Actually not sure if the chips in them are V1 or not.) No retail consoles are V1 and even good emulators will not support V1 display lists. The fast64 support for V1 appears to be because it was in the GBI when that was ported to Python. I'm happy to remove it from the codebase if you are okay with that.

I'd like to hear more fast64 opinions on this but yeah I think we should make it deprecated

fast64_internal/f3d/f3d_writer.py

Yanis42 · 2023-10-16T17:53:48Z

fast64_internal/f3d/f3d_writer.py

+        return [vertexBuffer.index(v) for v in tri]
+
+    t = 0
+    while t < len(triangles):


I'm not a huge fan of whiles in general because of the infinite loop risk if you're not careful, but that's just my own tastes

The index can be incremented by one or two in a particular pass. Editing the index variable in a for loop in Python is frowned upon.

fast64_internal/utility.py

Yanis42 · 2023-10-16T17:57:34Z

fast64_internal/utility.py

+    x2, y2 = x ^ 0x7F, y ^ 0x7F  # 7F - x, 7F - y
+    z = z ^ 0x7F  # = 7F - x - y


are those comments dead code?

No, it is explaining what operation we're actually trying to do. Doing 0x7F - value on the RSP would take an extra instruction and use an extra temporary register compared to value ^ 0x7F. Clarified the comments.

Dragorn421 · 2023-10-19T11:10:02Z

Where is the f3dex3 repo?

Where are its changes (notably, the gbi changes) documented?

sauraen · 2023-10-21T22:38:20Z

@Dragorn421
F3DEX3 has been public this whole time, but it was on a branch of a fork so not very visible. It has now been moved to the HackerN64 org and is available here: https://github.com/HackerN64/F3DEX3 Note that it is not yet stable.

Development on the microcode and on fast64 must occur in parallel for three reasons:

Changes to fast64 are needed to create display lists which contain new or modified features, in order to test and debug those features in the microcode.
Even if this wasn't true and the microcode could be perfected and "released" before fast64 changes were started, the microcode wouldn't be useful to the community until the fast64 changes were finished.
If all fast64 changes were held on a branch until they were all done and the microcode was finished, the resulting PR would be absolutely massive and impossible to review.

Yanis42

seems good, haven't tested again yet but I'll do it soon!

Yanis42 · 2023-10-21T22:54:00Z

fast64_internal/f3d/f3d_gbi.py

-    "LIGHT_8": 8,
-}
+# 1-8 for F3DEX2 etc., 1-10 for F3DEX3
+lightIndex = {"LIGHT_" + str(n): n for n in range(1, 11)}


this could use f-strings

Yanis42 · 2023-10-26T11:44:53Z

the import still seems broken (tried to import kokiri forest)

Traceback (most recent call last):
  File "C:\Users\Yanis\AppData\Roaming\Blender Foundation\Blender\3.6\scripts\addons\fast64-F3DEX3Basics\fast64_internal\oot\scene\operators.py", line 118, in execute
    run_ops_without_view_layer_update(parseSceneFunc)
  File "C:\Users\Yanis\AppData\Roaming\Blender Foundation\Blender\3.6\scripts\addons\fast64-F3DEX3Basics\fast64_internal\oot\scene\operators.py", line 33, in run_ops_without_view_layer_update
    func()
  File "C:\Users\Yanis\AppData\Roaming\Blender Foundation\Blender\3.6\scripts\addons\fast64-F3DEX3Basics\fast64_internal\oot\scene\operators.py", line 41, in parseSceneFunc
    parseScene(settings, settings.option)
  File "C:\Users\Yanis\AppData\Roaming\Blender Foundation\Blender\3.6\scripts\addons\fast64-F3DEX3Basics\fast64_internal\oot\oot_level_parser.py", line 271, in parseScene
    sceneObj = parseSceneCommands(sceneName, None, None, sceneCommandsName, sceneData, f3dContext, 0, sharedSceneData)
  File "C:\Users\Yanis\AppData\Roaming\Blender Foundation\Blender\3.6\scripts\addons\fast64-F3DEX3Basics\fast64_internal\oot\oot_level_parser.py", line 327, in parseSceneCommands
    roomObjs = parseRoomList(sceneObj, sceneData, roomListName, f3dContext, sharedSceneData, headerIndex)
  File "C:\Users\Yanis\AppData\Roaming\Blender Foundation\Blender\3.6\scripts\addons\fast64-F3DEX3Basics\fast64_internal\oot\oot_level_parser.py", line 430, in parseRoomList
    roomObj = parseRoomCommands(
  File "C:\Users\Yanis\AppData\Roaming\Blender Foundation\Blender\3.6\scripts\addons\fast64-F3DEX3Basics\fast64_internal\oot\oot_level_parser.py", line 521, in parseRoomCommands
    parseMeshHeader(roomObj, sceneData, meshHeaderName, f3dContext, sharedSceneData)
  File "C:\Users\Yanis\AppData\Roaming\Blender Foundation\Blender\3.6\scripts\addons\fast64-F3DEX3Basics\fast64_internal\oot\oot_level_parser.py", line 555, in parseMeshHeader
    parseMeshList(roomObj, sceneData, meshListName, roomShapeIndex, f3dContext, sharedSceneData)
  File "C:\Users\Yanis\AppData\Roaming\Blender Foundation\Blender\3.6\scripts\addons\fast64-F3DEX3Basics\fast64_internal\oot\oot_level_parser.py", line 649, in parseMeshList
    meshObj = importMeshC(
  File "C:\Users\Yanis\AppData\Roaming\Blender Foundation\Blender\3.6\scripts\addons\fast64-F3DEX3Basics\fast64_internal\f3d\f3d_parser.py", line 2255, in importMeshC
    parseF3D(data, name, transformMatrix, name, name, "oot", drawLayer, f3dContext, True)
  File "C:\Users\Yanis\AppData\Roaming\Blender Foundation\Blender\3.6\scripts\addons\fast64-F3DEX3Basics\fast64_internal\f3d\f3d_parser.py", line 1903, in parseF3D
    f3dContext.processCommands(dlData, processedDLName, dlCommands)
  File "C:\Users\Yanis\AppData\Roaming\Blender Foundation\Blender\3.6\scripts\addons\fast64-F3DEX3Basics\fast64_internal\f3d\f3d_parser.py", line 1536, in processCommands
    parseVertexData(dlData, vertexDataName, self)
  File "C:\Users\Yanis\AppData\Roaming\Blender Foundation\Blender\3.6\scripts\addons\fast64-F3DEX3Basics\fast64_internal\f3d\f3d_parser.py", line 1959, in parseVertexData
    vertexData = [
  File "C:\Users\Yanis\AppData\Roaming\Blender Foundation\Blender\3.6\scripts\addons\fast64-F3DEX3Basics\fast64_internal\f3d\f3d_parser.py", line 1973, in <listcomp>
    math_eval(match.group(10), f3d),
IndexError: no such group

sauraen · 2023-10-29T18:01:59Z

Fixed import issues, OoT scene import now works correctly. Also a mesh with materials with vertex colors only, normals only, and both / packed normals exports and re-imports with colors and normals preserved correctly.

Yanis42

just a small thing I noticed, now I'll try on Blender to see if I get any issues

Yanis42 · 2023-11-05T14:28:34Z

fast64_internal/f3d/f3d_parser.py

+            mat.rdp_settings.g_attroffset_st_enable = bitFlags & self.f3d.G_ATTROFFSET_ST_ENABLE != 0
+            mat.rdp_settings.g_attroffset_z_enable = bitFlags & self.f3d.G_ATTROFFSET_Z_ENABLE != 0
+            mat.rdp_settings.g_packed_normals = bitFlags & self.f3d.G_PACKED_NORMALS != 0
+            mat.rdp_settings.g_lighttoalpha = bitFlags & self.f3d.G_LIGHTTOALPHA != 0
+            mat.rdp_settings.g_ambocclusion = bitFlags & self.f3d.G_AMBOCCLUSION != 0
+            mat.rdp_settings.g_fresnel = bitFlags & self.f3d.G_FRESNEL != 0


if those values are only 0 or 1 would it be worth to use true/false? unless you plan on adding more possible values in the future

The objects on the left are bools (actually BoolPropertys), on the right it's checking if bits in an integer are set. This is the same way all the existing geometry mode properties are set.

Yanis42 · 2023-11-05T17:26:45Z

everything seems fine!

sauraen added 6 commits October 8, 2023 21:27

Porting former CustomMicrocode branch to latest fast64 and latest F3D…

e8686ee

…EX3 changes

Finished import of old changes

98aa706

Ported geom UI working

6693471

Removed passing around GBI type and hardware V1 flag

eee76f5

Fixing bugs in F3DEX3 import

23f03ed

Fixed bugs in F3DEX3 import

94c1bd6

sauraen added enhancement New feature or request f3d Has to do with the "f3d" code common to all games waiting for author Waiting for the author to answer questions, do changes, ... labels Oct 15, 2023

sauraen added 2 commits October 14, 2023 17:39

Fixed presets

8358194

blacked

f7b0c44

sauraen added review code please Ask that some other Fast64 dev reviews the code and removed waiting for author Waiting for the author to answer questions, do changes, ... labels Oct 15, 2023

Yanis42 reviewed Oct 16, 2023

View reviewed changes

sauraen added 5 commits October 16, 2023 21:27

Addressed Yanies comments

20ffb23

Additional type hints

2fe441f

blacked

20e2149

Fixed some missing F3DEX3 checks

e62c405

Removed hardware v1 support, cleaned up some othermode code

9a67d80

Fixes

a13b3de

sauraen requested a review from Yanis42 October 21, 2023 22:38

Merge remote-tracking branch 'origin/main' into F3DEX3Basics

37575ed

Yanis42 reviewed Oct 22, 2023

View reviewed changes

sauraen added 3 commits October 28, 2023 17:51

Converted some things to f-strings

a0a38ad

Fixed import issues

000ff56

Fixed bug with normals import

1069f27

sauraen requested a review from Yanis42 October 29, 2023 18:00

Yanis42 reviewed Nov 5, 2023

View reviewed changes

Yanis42 approved these changes Nov 7, 2023

View reviewed changes

Yanis42 approved these changes Nov 8, 2023

View reviewed changes

sauraen merged commit 042dc04 into Fast-64:main Nov 8, 2023

sauraen deleted the F3DEX3Basics branch November 8, 2023 15:42

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

First round of F3DEX3 changes for basic features #258

First round of F3DEX3 changes for basic features #258

sauraen commented Oct 15, 2023 •

edited

Loading

Yanis42 left a comment

Yanis42 Oct 16, 2023

sauraen Oct 17, 2023

Yanis42 Oct 17, 2023

Yanis42 Oct 16, 2023

sauraen Oct 17, 2023

Yanis42 Oct 16, 2023

sauraen Oct 17, 2023

Dragorn421 commented Oct 19, 2023

sauraen commented Oct 21, 2023 •

edited

Loading

Yanis42 left a comment

Yanis42 Oct 21, 2023

sauraen Oct 29, 2023

Yanis42 commented Oct 26, 2023

sauraen commented Oct 29, 2023

Yanis42 left a comment

Yanis42 Nov 5, 2023

sauraen Nov 7, 2023 •

edited

Loading

Yanis42 commented Nov 5, 2023

		x2, y2 = x ^ 0x7F, y ^ 0x7F # 7F - x, 7F - y
		z = z ^ 0x7F # = 7F - x - y

First round of F3DEX3 changes for basic features #258

First round of F3DEX3 changes for basic features #258

Conversation

sauraen commented Oct 15, 2023 • edited Loading

Yanis42 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Dragorn421 commented Oct 19, 2023

sauraen commented Oct 21, 2023 • edited Loading

Yanis42 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Yanis42 commented Oct 26, 2023

sauraen commented Oct 29, 2023

Yanis42 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sauraen Nov 7, 2023 • edited Loading

Choose a reason for hiding this comment

Yanis42 commented Nov 5, 2023

sauraen commented Oct 15, 2023 •

edited

Loading

sauraen commented Oct 21, 2023 •

edited

Loading

sauraen Nov 7, 2023 •

edited

Loading