[solved] Animation - Convert Matrices (model to local)

[majidemo]

I'm trying to animate my model with Three.JS (see documentation), the animation file (see file) I'm using only stores the bone transforms in modelspace 4x3 matrices (or 3x3 rotation matrix + position vector3).

I'm guessing Three.JS needs it to be in localspace because it doesn't work if I use the matrices directly.

Problem is: I have no idea how to convert it. If anyone can help me understand the mathematical formulas and the whys, that would be great.

The following images are the expected results:


The actual result (problem) can be found on this fiddle = https://jsfiddle.net/do4037kb/. (Note: you can move the camera around the model, and zoom)

I have also tried: (but still didn't work)

Code: Select all

var boneMatrix = root.getObjectByName(key).matrix;
frameMatrix.multiply(boneMatrix);
var inverseParentMatrix = new THREE.Matrix4();
inverseParentMatrix.getInverse(root.getObjectByName(key).parent.matrix);
frameMatrix.multiply(inverseParentMatrix);

Any idea how I can fix my script? (see fiddle for code and result)

[shakotay2]

Any idea how I can fix my script? (see fiddle for code and result)

umpf, well, not at a first glance.
I'd care for the transitions first.
Did you notice that if you comment out
frameMatrix.copy(rotation)
the moving cape is processed properly?

If you set frameBones.length to 2 in this line
for (var y = 0; y < frameBones.length; y++)
you can see that the mesh is displayed properly.

mesh_ok.JPG

So I'd proceed step by step.

I'm looking for the part of your code which handles this (if required):

"while many games have the transform matrix in parent space, so you have to multiply the parent matrix by the child matrix to get the world [...] matrix. If it has multiple parents, all must be multiplied." (Bastien)

[majidemo]

The mesh actually is ok if you click on the "stop animate". The transformation to the bones is the one making it deformed.

I did notice that the cape looked right, when I commented out "frameMatrix.copy", but removing the rotation component of the animation wouldn't be right.

Also, changing "frameBones.length" to 2 would only get the first 2 bone animation matrices.

"while many games have the transform matrix in parent space, so you have to multiply the parent matrix by the child matrix to get the world [...] matrix. If it has multiple parents, all must be multiplied." (Bastien)

The quoted is great advice too. Although, I'm not sure which matrices I have to multiply, the original bone matrices from the file? or the calculated bone matrices on the scene? and how would I do it for the animation?

I remember MrAdults telling me something similar in this thread. He was talking about the animation matrices on my files:

4) Unlike model bones, which are in modelspace, animation transforms are assumed to be in local space, so relative to the parent in the hierarchy. In addition to that, the frame transforms in the file are transposed. So you want to rework your whole frame loop to look something like this:

Code: Select all

            for fi in range(0, numFrames): # frames
                modelSpaceTransforms = []
                modelSpaceInverseTransforms = []
                #first accumulate all of the transforms in modelspace
                for bi in range(0, numBones):
                    boneMat = bones[bi].getMatrix()
                    frameMat = NoeMat43.fromBytes(ani.readBytes(48))
                    modelSpaceMat = boneMat * frameMat.transpose()
                    modelSpaceTransforms.append(modelSpaceMat)
                    modelSpaceInverseTransforms.append(modelSpaceMat.inverse())
                #now run through and calculate local space transforms to use as the animation transforms
                for bi in range(0, numBones):
                    localSpaceMat = modelSpaceTransforms[bi]
                    if bones[bi].parentIndex >= 0:
                        localSpaceMat = localSpaceMat * modelSpaceInverseTransforms[bones[bi].parentIndex]
                    animFrameMats.append(localSpaceMat)

But I'll investigate further with the idea's you've given. And as always, I can't thank you enough for the help

[majidemo]

I found this out: DirectX and OpenGL uses different conventions in dealing with matrix.

I read about it here: http://www.mindcontrol.org/~hplus/graph ... ayout.html

Traditional mathematicians, and OpenGL, tends to prefer colum vectors. Meanwhile, certain other legacies of computer graphics, as well as DirectX, tend to prefer row vectors, on the left. The confusion gets even more complete when you start talking about "pre-multiplying" and "post-multiplying" matrices. Pre-multiplying may mean multiplying it on the left (if you're the row vector type), or it may mean multiplying it on the right (if you're the OpenGL type) -- if by "pre" you mean that the pre-multiplied operation happens before the target matrix operation. If instead you mean, with pre-multplying, that the matrix you're pre-multiplying goes on the left, then it means that it happens afterward in the OpenGL notation, but it still means that it happens before in the DirectX notation

Considering that the game I'm working with uses DirectX, it is understandable that the matrices stored in the files follows DirectX convention... Three.JS on the otherhand uses WebGL or OpenGL ES 2... A different convention...

I have updated the fiddle but still not yet completely fixed, but seems to be a bit closer: https://jsfiddle.net/do4037kb/21/

This is the part where I tried to multiply the frame matrix with the bone and its parent bone.

Code: Select all

var modelSpaceMatrix = new THREE.Matrix4();
modelSpaceMatrix.multiplyMatrices(root.skeleton.bones[y].userData.modelMatrix, frameMatrix);

inverseFramematrices.push(new THREE.Matrix4().getInverse(modelSpaceMatrix));

var parentId = root.skeleton.bones[y].userData.parentId;

if (parentId >= 0) {
  modelSpaceMatrix.multiplyMatrices(modelSpaceMatrix, inverseFramematrices[parentId]);
}

-------------------------

This is another good read but it's the opposite of what I wanted: http://gamedev.stackexchange.com/questi ... to-directx
So I guess, transposing the matrix should fix the directx/opengl problem.

-------------------------

Here is an example exporter from Blender into ThreeJS format: https://github.com/mrdoob/three.js/blob ... ion.py#L56
There should be something interesting there.

[majidemo]

I finally figured this out. What MrAdults and Bastien said helped me get the answer. Thanks again, sir shakotay2.

#closed

[shakotay2]

that's great!

Could you please give a brief survey about the main points to get your solution?
Or your final code.

Because there was transposing, multiplying and inversing matrices.

Transposing was only required to adapt the opengl matrices for DirectX, right?

[majidemo]

That is right, I had to transpose the matrices to make it work with WebGL/OpenGL.

The rest, like the multiplying and inverting was the part that converted the modelspace bones to localspace (heirarchy). I'm not very good in explaining, so I hope showing you the code will suffice

Code: Select all

var frameMat = new THREE.Matrix4();
frameMat.set(
  bone.rotationMatrix[0], bone.rotationMatrix[3], bone.rotationMatrix[6], bone.positionVector[0],
  bone.rotationMatrix[1], bone.rotationMatrix[4], bone.rotationMatrix[7], bone.positionVector[1],
  bone.rotationMatrix[2], bone.rotationMatrix[5], bone.rotationMatrix[8], bone.positionVector[2],
  0.0,     0.0,     0.0,     1.0
);

// var boneMat = new THREE.Matrix4().copy(_bone.userData.modelMatrix);
var boneMat = _bone.matrixWorld;

var modelSpaceMat = new THREE.Matrix4();
modelSpaceMat.multiplyMatrices(frameMat, boneMat);
_bone.userData.modelSpaceMat = modelSpaceMat;

var localSpaceMat = new THREE.Matrix4();
localSpaceMat.copy(modelSpaceMat);

if (parentId >= 0) {
  var inverseModelSpaceMat = new THREE.Matrix4().getInverse(_bone.parent.userData.modelSpaceMat);

  localSpaceMat.multiplyMatrices(inverseModelSpaceMat, modelSpaceMat);
}

1. I had to transpose the rotation part of frameMatrix.
2. Multiply the boneMatrix to the frameMatrix. (had to swap the factors so: frameMat * boneMat)
3. If the bone has a parent I also had to multiply the result of frameMat*boneMat to the parents inverted frameMat*boneMat.
4. The result is the localspace matrices.

[shakotay2]

Thank you very much!
Great to see your progress; seems you're walking a steep learning curve.

[majidemo]

True, been a long journey getting here. First time reversing a file format was the hardest, but thanks to your help I've progressed far enough.

Now the only files left to unpack/extract are the weapons/shield and map datas like: stones, trees, grass...as the terrain was already completed (still with your help ;D). Characters, Npc's, monsters and other data are also done.

I'll open another topic about the weapons (soon, if I still can't figure it out by myself), as I'm a bit confused about it. The weapons obj file doesn't seem to have any indices data.

Cheers!