MMD tools features, and features which are not yet implemented

[Hogarth-MMD]

Thank you and congratulations to @powroupi and to @lordscales91 for your excellent work in debugging mmd_tools. I am trying to make a list of features of mmd_tools, so if someone can please review this list of MMD tools features and correct or fill in the blanks for me and also for other people, that would be great:

MMD tools features:

Ability to pose or animate these morphs of an imported PMX model in Blender:
material morphs No.
UV morphs No.
bone pose morphs No.
group morphs No.

The data of these morphs is preserved and will be re-exported when a previously imported PMX model is exported from Blender:
material morphs - Yes.
UV morphs - Yes (dev_test by powroupi), No (master.zip).
bone pose morphs - Yes (dev_test by powroupi), No export (master.zip).
group morphs - Yes (dev_test by powroupi), No (master.zip).

Dual/bilingual naming of shape keys is preserved and will be re-exported when a previously imported PMX or PMD model is exported from Blender Yes. (with "Morph Tools" panel)

Imported shape keys can be edited, renamed, re-ordered and the re-exported from Blender with their data intact. Yes. (with "Morph Tools" panel)

Dual/bilingual naming of PMD and PMX bones and PMX materials is preserved and will be re-exported when a previously imported PMX or PMD model is exported from Blender Yes.

Correct display of an MMD model's ambient color in Blender. No.
Correct display of an MMD model's toon textures in Blender. No.
Correct display of an MMD model's sphere textures in Blender. Yes. (except "subtract blend" type)
Ability to use .fx effects in Blender. No.

Ability to import SDEF in Blender. No, SDEF is converted to BDEF.

Ability to import .vmd morphs animation. Yes.
Ability to import .vmd bones animation. Yes.
Ability to import .vmd camera animation. Yes.
Ability to import .vmd lamp animation. Yes.

Ability to export .vmd files from Blender. No.

Ability to import and re-export physics objects correctly. Yes (at least no bugs have been found yet).
Physics animation. Yes.

My recent deviantart journal entry:
MMD models in Blender
Some links to help people get started with importing, exporting, editing, converting or animating MMD models in Blender:
http://hogarth-mmd.deviantart.com/journal/MMD-models-in-Blender-626045612

[powroupi]

Simply reply 😃 :

The data of these morphs is preserved and will be re-exported when a previously imported PMX model is exported from Blender:

UV morphs - Yes (dev_test), No (master)
bone pose morphs - Yes (dev_test), No export (master)
material morphs - Yes
group morphs - Yes (dev_test), No (master)

Dual/bilingual naming of shape keys is preserved and will be re-exported when a previously imported PMX model is exported from Blender ?

Yes (with "Morph Tools" panel)

Imported shape keys can be edited, renamed, re-ordered and the re-exported from Blender with their data intact. ?

Yes (with "Morph Tools" panel)

Correct display of an MMD model's sphere textures in Blender. ?

Yes (except "SubTexture" type)

Ability to import and re-export physics objects correctly. ?

Possibly yes (i haven't found any mistake)

[Hogarth-MMD]

Thanks so much for the fast reply, @powroupi ! Really impressive work on MMD tools! Now I have re-edited the list of features in my first post, adding all of your corrections into it.

[Hogarth-MMD]

The remaining programming tasks are very difficult tasks, except VMD export. VMD export shouldn't be too difficult. We already have the code for VMD import. VMD export would just be the reverse of VMD import.

[nathanvasil]

@powroupi

Not sure if you're looking for projects... I noticed:

"Correct display of an MMD model's ambient color in Blender. No.
Correct display of an MMD model's toon textures in Blender. No.
Correct display of an MMD model's sphere textures in Blender. Yes. (except "subtract blend" type)
Ability to use .fx effects in Blender. No."

and these could be addressed. The default MMD shader could be recreated in GLSL (should be a straight forward port), and I could write it, if Blender provided the GLSL code with the proper parameters.

Widespread effect compatibility would be more work, but it should be possible-- MME code is open source, if I remember correctly, and GLSL and HLSL are similar enough that automated translation shouldn't be too hard (as demonstrated with projects like https://github.com/aras-p/hlsl2glslfork ).

"Ability to import SDEF in Blender. No, SDEF is converted to BDEF."

I can't help with this, but I wanted to mention that SDEF is one of the few remaining things with MMD Tools that makes me sad (almost everything else is fantastic). Would it be possible to remember and restore SDEF in the same way that MMD Tools remembers and restores so much else? Would it be possible for MMD Tools to create volume preservation morphs to apply on bone manipulation to make up the difference between SDEF and BDEF when animating in Blender?

[powroupi]

@nathanvasil
About GLSL, i'm not sure when will blender support GLSL code directly, we use node editor to make custom shader in blender, but it would be a little complex to use. And it would be great if you can help with GLSL code then. 😄

About SDEF, i'm not sure about the math of it, and there are 4 parameters saved to a vertex (Weight, C, R0, R1), but I couldn't find a easy way to store those values (C, R0, R1) in blender. It would be great if someone can solve it. 😄

[nathanvasil]

I found https://en.wikibooks.org/wiki/GLSL_Programming/Blender/Minimal_Shader . I haven't actually done anything with it, but it does look like there should be a way to get a vertex and pixel/fragment shader into Blender.

I don't understand the SDEF math either, which would make it difficult to create compensatory shape keys. but I believe MMD Tools already uses a few techniques that would make it possible to store SDEF. R0, R1, and C0 are 3D coordinates that tend to exist in a limited number of discrete values, and could be stored by the positions of shadow bones, with SDEF vertices assigned to these groups (with 0 weight). While not always true, these values tend to be shared by many SDEF vertices, and if you checked for duplicates, you wouldn't end up making ten thousand new vertex groups. If there was any way to hide these temporary vertex groups, that would be ideal, but anybody trying to preserve SDEF shouldn't be considering any significant weight painting in Blender anyways. (Weight, of course, doesn't change with the conversion between SDEF and BDEF2, and could be stored as it always is.)

[ptrthomas]

@nathanvasil just trying to understand SDEF and BDEF better here, do you have a link to some reading material ?

Meanwhile for animation, I'm thinking that switching to Unreal Engine (if you have a decent PC) is the best option. Here's my latest video showcasing cloth physics: https://www.youtube.com/watch?v=fd7KumzdsTw

And - if I understand the purpose of SDEF right, this technique can be used in Unreal Engine to solve for the same thing, kind of a hack but still: https://docs.unrealengine.com/latest/INT/Engine/Animation/NodeReference/SkeletalControls/TwistCorrective/

[nathanvasil]

@ptrthomas sorry, I don't have any links-- everything I know is just from experimentation.

Unreal doesn't look like it's using SDEF. Instead, it's using per-axis bone rotation to drive any kind of node. Typically, that would be volume-correcting shape key/morph. That's a good way for it to handle the problem, and useful for other things as well (an example of creative use would be driving a blink morph based on X axis rotation of the eyes, to automate eyelid droop when looking down).

(Personally, I'm still fine with rendering in MMD. There's not a lot you can't do, not if you're willing to figure out how to make Blender set it up and willing to write your own shaders. Still, great video, very cool!)

[ptrthomas]

@nathanvasil thanks, and the eyelid droop example is appreciated, gives me some ideas.

Probably obvious by now, but I mean to say Blender is a core part of my workflow. And agree that if you are into just rendering single frames, is perfect. When it comes to animation and environment / particle effects, that is something I think MMD / MMM etc can never reach a certain level. Cloth would be a good example. In theory you could write your own shader in UE too.

[powroupi]

@nathanvasil I found https://en.wikibooks.org/wiki/GLSL_Programming/Blender/Minimal_Shader...

Yeah, i know that, but it is for blender's game engine, and shaders can only be viewed when you play your scene. I hope blender can support that directly in 3d view. 😊

I just tested C, R0, R1 for several models, the counts cannot be reduced to a reasonable value when simply check duplicates. Counts of C are easily over hundreds (sometimes thousands), counts of R0, R1 are sometimes over tens. 😢 (need more key informations to reduce it on import and restore back on export)

[lordscales91]

I couldn't find a easy way to store those values (C, R0, R1) in blender. It would be great if someone can solve it.

We could use the same technique we are using to store the names of the vertex morphs. Storing those values as properties in the mmd_root. However, this case is a bit more complicated since those values are assigned to individual vertexes, and Blender doesn't provide any option to store a reference to individual vertexes. We could use vertex groups as @nathanvasil suggested, but that would mean having a vertex group per each vertex with SDEF, which will bloat up the amount of vertex groups.

@ptrthomas Awesome video as usual. I also think that the future is moving to Unreal Engine or Unity. However, I don't like the idea of leaving the MMD Community behind and, at the same time, "ripping off" their work (models, motions, stages...). I love this blender addon so much because it has both import and export features. This allows us to get something from MMD (a model base for example) and give something back (a new model).

Maybe we could work on a special addon to create a 2-way bridge between UE and MMD.

[nagadomi]

As far as I know, the original algorithm of SDEF is from mikoto/keynote(bone addon for metasequoia). mqdl (author of keynote) described SDEF at http://d.hatena.ne.jp/mqdl/20080525/1211706244 (japanese).

[lordscales91]

@nagadomi Thanks for the reference. But it would be more useful if someone translates it to English. Because Google Translate will not be very helpful here due to technical language (It's describing an algorithm)

[ptrthomas]

@lordscales91 I don't particularly appreciate the obsession and drama in some parts of the MMD community for making it hard for "outsiders" to get hold of models and motion data. I'm actually tempted to upload all the UE artifacts I create password protected with clues in the form of references to Shakespeare or Hollywood movies. And then later throw a tantrum saying someone "misused" my model and disable downloads.

Okay, jokes apart - if you look at the volume of MMD videos being uploaded on YouTube nowadays, I don't see any danger of the MMD community being "left behind". Actually the same old choreography and camera motion is getting really boring, we need to move forward, and I think UE is the answer.

@nagadomi thanks for the link.

P.S. If anybody happens to have the motion files for "Mr. Music" (or knows how to get it) that would be very much appreciated.

[powroupi]

@nagadomi Thanks for the reference. If i understand it right, C is a generated (calculated) value that may not be necessary to store in blender. :) (but i don't understand the deformation algorithm >_<)

From my understand:

• P: a vertex
• C: the position that P is projected to the bone
• R0: the position that the bone intersect with lower (Bone1) box
• R1: the position that the bone intersect with upper (Bone0) box
• B: the tail position of Bone0 / the head position of Bone1 (assume Bone1 is connected to Bone0)

EDIT:
Google search for algorithm spherical blend skinning or dual quaternion skinning 😄

[nagadomi]

Sorry, I am not understanding this algorithm and my english writing skill is very poor.
(誰か翻訳できない？)
But I tried to translate it.
Here is a free translation.
+++
First, parameters of born is not managed with matrix. That's managed with a quaternion for rotation, a vector for scaling and a vector for movement. This is also known as SRT.

Please see the figure.

P: a vertex on mesh which is a skinning target.
C: a point that is on perpendicular line from P to bone.

P is rotated around the C using a quaternion that is obtained by multiplying the weight of born0 and the weight of bone1.
Note that the center of this operation is not B.

This deformation technique is called Spherical Blend Skinning or Dual quaternion skinning.
This deformation reduces mesh collapsing at joints when they rotate, but sometimes, mesh to swell.
To fix this issue, we need to interpolating C. In keynote, we uses linear interpolation for a vector B->R0 and a vector B->R1.

[nathanvasil]

@powroupi

"Yeah, i know that, but it is for blender's game engine, and shaders can only be viewed when you play your scene. I hope blender can support that directly in 3d view."

Oh, I didn't realize that. That's a shame. It would be an obvious way to improve Blender, and hopefully somebody more talented than me will take on that job eventually :)

"Counts of C are easily over hundreds (sometimes thousands), counts of R0, R1 are sometimes over tens."

Thank you for testing. I hope it wasn't too much trouble; I of course never actually counted out the values, but just noticed that there were a lot of duplicates.

I will try to look into a lot of the information that nagadomi provided. That's all very interesting stuff @nagadomi , thanks. If I am able to puzzle anything out (I always need some reading material for family TV time), I will share it.

@ptrthomas If you want to get mischievous with rules (I feel the same way: I release everything as public domain as I can, considering the source material), then look into copyleft/share-alike licenses. They have a nice way of infecting everything they touch and destroying rules.

[powroupi]

Hi, all, i guess i found a solution to preserve SDEF data. I just create 3 shape keys, say mmd_sdef_c, mmd_sdef_r0, mmd_sdef_r1, then it becomes pretty simple to implement, i don't even need to understand SDEF. ☺️

Maybe we can add a tool to configure that in blender when we understand it more, but i think it is easier to do that in PMDE/PMXE. :)

[lordscales91]

@powroupi That's a nice approach to store the SDEF data. You are very smart 😆

@nathanvasil copyleft/share-alike licenses are not supposed to be used as a weapon to destroy rules. You can't license a derived work based on someone else's work (like a model base) without his/her explicit approval(considerations for licensors: ... Licensors should also secure all rights necessary). Unless that work is also licensed with a license of this kind. Which is not the case for most (if not all) of the MMD resources out there.

[nathanvasil]

@lordscales91 I'm not talking about using copyleft to destroy rules on anything already created; I'm talking about using it to destroy rules (other than any copyleft provisions themselves) on future works. Of course, this requires making something good enough that people want to use it, but that includes videos as well as models.

With most MMD resources, permission to create derivative renders or models is granted by the permission to use or edit respectively. In the US, a judge examining a case would look at the context in order to interpret those permissions, which are not typically written for legal clarity. I believe that a judge would find that all rights to derivative MMD works go to the derivative author, in the absence of anything written to the contrary. But of course, there will never be any cases.

[nathanvasil]

Had some chance to test SDEF preservation. Looks good to me. Thanks!

[Hogarth-MMD]

Maybe mmd_tools will never be 100% feature complete. But we are getting c;lose to the stage of mmd_tools being able to do a 100% import of MMD models into Blender. 😄

[powroupi]

Newer discussion -> #145