MSET RESEARCH PROJECT + FILES UPLOADED + SOURCES

[Xemnass]

Interesant, can you give me a link?

Maybe I can disassemble it =D

[Xemnass]

I've attached the KH2DumperX disassembled in ASM =D

[Mirrorman95]

That looks great, but there a few unresolvable compiling errors when trying to put that back into an exe. Do you have any idea what I'm doing wrong?

[Xemnass]

Can you post a log?
Maybe do you miss some dependencies?
Or simply, when you reapcj the exe, I thinmk that the program have to "execute" so, maybe have to saty even all the necessary to do the "unpack" even if the compilations wont unpack nothing, i think...

Did you have the Redistributable C++ pack for you enviroment as say in the blog?:
http://xeeynamo.blogspot.com/2011/07/ki ... l-mix.html

[Mirrorman95]

I tried to disassemble it too, but reassembly gave me another string of errors, so it may just be my compiler, Masm32, and not your code that is giving me a problem. This is its disassembly: http://www.majhost.com/gallery/mirrorma ... disasm.txt

[Shapyi]

I was wondering, have any of you been able to get any KH2 animations exported? Because if you're looking for a quick solution you can just get them from kkdf2's viewer. You don't even need the source code to do it (although it makes it easier). Inserting a little code into his viewer using .NET Reflector does the trick.

Here is a video of Sora is the DirectX *.x format running around in my XNA game engine:

http://www.youtube.com/watch?v=Lh0tUgORu34

Let me know if you guys have a better method.

[ultimaespio]

Pretty impressive. Just out of curiosity, what did you use to export the model? If they match with the Noesis exports, we have a winner.

[Mirrorman95]

I was wondering, have any of you been able to get any KH2 animations exported? Because if you're looking for a quick solution you can just get them from kkdf2's viewer. You don't even need the source code to do it (although it makes it easier). Inserting a little code into his viewer using .NET Reflector does the trick.

Here is a video of Sora is the DirectX *.x format running around in my XNA game engine:

http://www.youtube.com/watch?v=Lh0tUgORu34

Let me know if you guys have a better method.

Sweet freaking jebus. You did it! By the way, what code did you insert?

If this really works, we either need to import this to Noesis or polish it off with Yaz0r's program if we can, because yaz0r's runs imperfectly when it comes to ANB facial animations, while kkdf2's does not. (I used anb2mset to convert those ANBs.)

[SirLoon]

I tried to disassemble it too, but reassembly gave me another string of errors, so it may just be my compiler, Masm32, and not your code that is giving me a problem. This is its disassembly: http://www.majhost.com/gallery/mirrorma ... disasm.txt

reassembly means you recompile whole code with bunch of garbade from c linker and you dont recognize what is user code itself and whats inserted by compiler

[Shapyi]

I wrote my own kh2 model viewer/exporter a few years ago with kkdf2's help. I'm not going to release it because its a mess. But I will go over the procedure I used to get animations exported and hopefully you guys can get it working. It's a bit of a process but its a lot easier than reverse engineering the animation system. If you're familiar with mdlx models this shouldn't be too bad.

NOTE: All my code is done in Visual C# Express Edition 2008.

First step is to modify kkdf2's viewer to export matrix transforms for each frame. This can be done by adding some code to his program either using his source code or .NET Reflector (I've done both methods). Here is a the latest version I modified with .NET Reflector:

http://www.mediafire.com/?ck31xco38hdk6ti

When you push "Bind Controller" AND an animation is selected it will dump the transforms into a data file (*.dat). Otherwise it will crash. Here is the code I used to make that data file:

Code: Select all

private void DumpMatrixTransforms()
{
	// Generate the file name
	string filename = listView1.SelectedItems[0].Text;
	string name = listView1.SelectedItems[0].Text;
	string index = listView1.SelectedIndices[0].ToString();

	filename = filename.Replace('#', '_');
	
	filename = filename + "_" + index + ".dat";

	if (filename.Length < 5)
	{
		filename = "mat-dump.dat";
	}

	// Create a new memory stream
	MemoryStream mat_data = new MemoryStream();
	BinaryWriter mat_writer = new BinaryWriter(mat_data);
	FileStream outfile = File.OpenWrite(filename);
	int max_ticks = 1;

	// Get the frame count (ticks)
	foreach (ListViewItem lvi in listView1.SelectedItems)
	{
		decimal temp = (decimal)((MotInf)lvi.Tag).maxtick + numericUpDownStep.Value;
		max_ticks = (int)temp;
		break;
	}

	mat_writer.Write((int)_Sora[0].Ma.Length); // bone count (4 bytes)
	mat_writer.Write((int)max_ticks); // frame (tick) count (4 bytes)

	// output all the matrix transforms for each frame
	for (int i = 0; i < max_ticks; i++)
	{
		tick = i;
		recalc(); // this tell his viewer to calculate the animation for the frame

		// output each matrix for each bone (matrix4x4 * 4 bytes)

		foreach (Matrix mat in _Sora[0].Ma)
		{
			/* Matrix Format
				[M11][M12][M13][M14]
				[M21][M22][M23][M24]
				[M31][M32][M33][M34]
				[M41][M42][M43][M44]
			*/
			mat_writer.Write((float)mat.M11); mat_writer.Write((float)mat.M12); mat_writer.Write((float)mat.M13);
			mat_writer.Write((float)mat.M14);

			mat_writer.Write((float)mat.M21); mat_writer.Write((float)mat.M22); mat_writer.Write((float)mat.M23);
			mat_writer.Write((float)mat.M24);

			mat_writer.Write((float)mat.M31); mat_writer.Write((float)mat.M32); mat_writer.Write((float)mat.M33);
			mat_writer.Write((float)mat.M34);

			mat_writer.Write((float)mat.M41); mat_writer.Write((float)mat.M42); mat_writer.Write((float)mat.M43);
			mat_writer.Write((float)mat.M44);
		}
	}
	
	// Output all the bytes to a file
	byte[] b_data = mat_data.ToArray();
	
	foreach (byte b in b_data)
	{
		outfile.WriteByte(b);
	}

	outfile.Close();

	MessageBox.Show("Animation transforms dumped.", this.Text, MessageBoxButtons.OK, MessageBoxIcon.Information);
}

This outputs the number of bones (4 bytes), then the amount of frames (4 bytes), and finally it outputs all the 4x4 matrix transforms. I'm providing the source code so you can see how I created the file, it will make it easier to parse later. On to the fun part

kkdf2 took a chunk of code from pcsx2 and emulated it in his viewer to get animations working. In order to get the models into the DirectX format I needed to give it the data, when calculated, arrived at the same result as if PS2 calculated it. In other words, work backwards. For each frame, each bone has a local matrix transform that gets calculated to an absolute transform through the bone hierarchy. Here is the formula, this is how meshes are animated with the format I'm using:

Code: Select all

bone.absoluteMatrix = bone.localMatrix * parentBone.absoluteMatrix

PS2 gives us essentially the same result but using a far more complex formula (IK solvers, etc). All we need is some basic linear algebra to solve for the local matrix. C# has Managed DirectX which provides matrix and vector functions. I came up with the formula below:

Code: Select all

bone.localMatrix = bone.absoluteMatrix * Invert(parentBone.absoluteMatrix)

Now when the animation system in DirectX calculates the transforms for each frame it will arrive at roughly the same result as PS2. For those familiar with mdlx, each bone as a few pieces of data but we really just need the parent index, absolute transform (from frame data) and local transform (calculated). For each frame, fill the absolute transform matrix with that value then calculate the local transform. Spit the locals into the model format of your choice. I treated the list of bones as a tree and did a level-order traversal to calculate the local transform for each. Here is the code:

Code: Select all

// Used to take absolute transforms provided and calculate the local matrix for each bone.
private void calc_local_mats(int index)
{
	AxBone ax_bone = boneList[index];

	// parent bone
	if (ax_bone.parent == -1)
	{
		ax_bone.local = ax_bone.absolute;
	}
	else
	{
		// localMatrix = absoluteMatrix * parent.absoluteMatrix.Invert()
		Matrix B = boneList[ax_bone.parent].absolute;
		Matrix C = ax_bone.absolute;

		B.Invert();

		boneList[index] = C * B;
	}

	// if the current bone is not a leaf
	if (!mesh.bones[index].leaf)
	{
		List<int> indices = new List<int>();

		// build a list of all its children
		for (int i = 0; i < mesh.bones.Count; i++)
		{
			if (index == mesh.bones[i].parent)
			{
				indices.Add(i);
			}
		}

		// calculate the locals for each child
		foreach (int i in indices)
		{
			calc_local_mats(i);
		}
	}
}

// this the function i used to determine if a bone is a leaf.
private bool joint_is_leaf(int index)
{
	for (int i = 0; i < mesh.bones.Count; i++)
	{
		if (mesh.bones[i].parent == index)
			return false;
	}

	return true;
}

That's basically it. To recap:

1. Dump the animation transforms.
2. Load that data file and parse it.
3. Fill a bone list with transforms, one frame at a time. Make sure there is parent index data
4. Perform a level order traversal, treating the list as a tree and calculate the local transform
5. Output the local transforms to the model format as a frame (preserving the hierarchy)
6. Repeat

I'm sure I'm missing some details so ask questions if you aren't clear on what I did. Essentially you are working backwards from the data PS2 calculates. This is not as "pure" as writing an IK solver similar to PS2 so its a quick and dirty solution that produces pretty nice results. You'll want to clean up these animations in Max or whatever package you use. There won't be "keyframes", there will be a frame for each frame of the animation

Again credit for kkdf2 for his viewer and help. I never shared this code with anyone but I'm glad I found this board. I hope you guys get some ideas from this and share any code you come up with. I'd be really happy to see it.

[SirLoon]

Respect ...

[Darkfox]

Excellent work there, Shapyi! Very happy to see animation support coming into play here. At current yours would seem to be the best for getting animations until direct MSET support happens in Rev's Noesis plugin.

Unfortunately pcsx2's debugger functionality is no longer operational, and even in older builds was only so useful. As such, even i don't think you will find an easily understandable form of the MSET IK processing. But hey, we can use all the help we can get.

I share your sentiments here, PCSX2 was my one-stop for general PS2 hacking which, without a debugging feature, is no longer possible. I wish they would release a version with better debugging features but that may not be for quite a while...

[Dinoguy1000]

PCSX2 is open source; if you have the time, you could always contribute towards better debugging support in it.

[Xemnass]

@Shapyi
Very, very, very, Impressive =D
You have my respect. XD
Howevr, you did a god job, now the still work, is to "pack" an MSET+MDLX plugin =D and the jobs is done =D
Very, very a god job, but i have a question I know that the MSET file of KH2 is a bit different from the MSET file of KH1, if I not go wrong right? Maybe we can find a solution to "pack" a MDLS+MSET plugin =D for do a "complete job" XD
Howver very nice and interessant post =D and Code =D

Respect =D

P.S.=
@Mirrorman95
I used IDA free think you can simply load my ask and recompile itself =D

[Xemnass]

Sorry for the double post, but now i was looking for my DUIumped KH models, and.....

I dindn't find any ANB files instead I find a.fm files???? why???