Tekken 6

[d2rnattakorn]

Thank you so much fxfx for this progress.
All the file from the dise are start from DATA301.BIN, DATA302.BIN, DATA303.BIN to DATA320.BIN
I only upload DATA312.BIN because it is the smallest file.

Please let us know how you go with DATA312.BIN.
And please let me know if you want me to upload any more file, I am happy to upload it for you fxfx.

Thank you

[fxfx]

I had a look over DATA312 and it appears to contain information on Customisation parts

There was a nice long list of ascii strings containing names ("Kint Cap", "Metal Headband", "Cowboy Hat"...) and also what appear to be directory heirarchies ("lar1/sunglass_04", "cmn1/tfm_cmnglow000", "cmn1/tfm_rageglow002", "cmn1/katana"...) in a different place

The name strings are listed in different languages (English, French, German and Russian) and do not contain any other information pointing to associated data.

But generally, the structure of the entire BIN file is still chaotic and not easy to extract without knowing whether the offsets are relative to their containers or some other arbitary point.

Both these BIN files contain similar data chunks:
"NDXR"
"NTXR"
"GID"
"BONE"

The "BONE" ones are the most intriguing because we would expect geometry information to be nearby.

The "NDXR" chunks appear to be the primary data-containers though, and it may be possible to extract them by a brute-force approach through a simple script.

Don't bother uploading anymore samples, these two BIN files should be more than enough

[shadowmoy]

I had a look over DATA312 and it appears to contain information on Customisation parts

There was a nice long list of ascii strings containing names ("Kint Cap", "Metal Headband", "Cowboy Hat"...) and also what appear to be directory heirarchies ("lar1/sunglass_04", "cmn1/tfm_cmnglow000", "cmn1/tfm_rageglow002", "cmn1/katana"...) in a different place

The name strings are listed in different languages (English, French, German and Russian) and do not contain any other information pointing to associated data.

But generally, the structure of the entire BIN file is still chaotic and not easy to extract without knowing whether the offsets are relative to their containers or some other arbitary point.

Both these BIN files contain similar data chunks:
"NDXR"
"NTXR"
"GID"
"BONE"

The "BONE" ones are the most intriguing because we would expect geometry information to be nearby.

The "NDXR" chunks appear to be the primary data-containers though, and it may be possible to extract them by a brute-force approach through a simple script.

Don't bother uploading anymore samples, these two BIN files should be more than enough

"NDXR" = geomerty (vertex,uvs, indices etc...)
"NTXR" = texture main chunk
"GID" = texture data
"BONE" = do i need to precise ?

[FurryFan]

Okay I had some time to look at the files:
To the best of my knowledge no 3d models are in the decompressed Data312 file, but there are models in the Data301 file.
I found 3 different types of models, one type had the following format:
Vertex(x,y,z) NormalMapping(x,y,z)

Another type had this:
Vertex(x,y,z) NormalMapping(x,y,z) Float, Float, Float,Float

The last type looked vaguely familiar to models (but are nowhere near the same) in the PS2 version of Tekken Tag Tournament:
Vertex(x,y,x) 4byte
Normal(x,y,z) 4byte
Texture(u,v) 4byte 4byte

All of the the three different types have some kind of "rogue" Face List, that I can not seem to decipher, that has many "FF FF" bytes scattered inside.
Also there is usually a xml file right before the models that says what the model is.
Here is a picture of the vertexes from some kind of motorcycle or car:

[Tosyk]

Okay I had some time to look at the files:
To the best of my knowledge no 3d models are in the decompressed Data312 file, but there are models in the Data301 file.
I found 3 different types of models, one type had the following format:
Vertex(x,y,z) NormalMapping(x,y,z)

Another type had this:
Vertex(x,y,z) NormalMapping(x,y,z) Float, Float, Float,Float

The last type looked vaguely familiar to models (but are nowhere near the same) in the PS2 version of Tekken Tag Tournament:
Vertex(x,y,x) 4byte
Normal(x,y,z) 4byte
Texture(u,v) 4byte 4byte

All of the the three different types have some kind of "rogue" Face List, that I can not seem to decipher, that has many "FF FF" bytes scattered inside.
Also there is usually a xml file right before the models that says what the model is.
Here is a picture of the vertexes from some kind of motorcycle or car:

Oh, grate work man, I'll stay tune!

[chrrox]

the faces use that as a reset for the strip.
just reset the tri-strip when you read that.

[MrAdults]

It's actually almost exactly the same as the Soul Calibur 4 format:

http://img31.imageshack.us/i/tekkenguy.jpg/

I'm as clueless as anyone else as far as knowing how to actually get data out of the decompressed bin intelligently, though. Guessing the file tables somewhere in the xex. The guy above is located at offset 253394944 (0xF1A8000) of the decompressed data301.bin.

Some of the files are actually the exact same format as SC4, but one of the reserved values in the NDXR header that is always 0 in SC4 now contains non-0 values to indicate version changes presumably, and non-0 version files have a couple differences in their vertex formats.

As far as I can tell, when version > 0, the UV+color array becomes only a UV array, so it has a 4 byte stride instead of 8, and the color values are removed from vertex types 6 and 7 as well, reducing their sizes to 24 and 40 (from 28, 44), respectively.

The other version > 0 change is that the index for the diffuse texture seems to be located 18 bytes into the texture data instead of 34 bytes. But I don't know how to intelligently parse that data. If I did, I might not have to worry about those hardcoded offsets.

Edit: I was completely wrong about the version flag. The existence of the color components depends on the low bits of the vertex type value. There are actually a few of these colorless components in SC4 models too, but I hadn't noticed them, because they're mostly tiny out of the way bits (and they didn't show up on any unweighted surfaces, apparently, which prevented the vertex stride from ever being wrong).

[FurryFan]

MrAdults, that is a good observation. The format for Tekken Tag Tournament is also very similar to Soul Caliber 3.

[MrAdults]

I endeavored to find out a bit more about the material command lists (or whatever the hell they really are) to get Tekken 6's textures all mapped out correctly:

http://img80.imageshack.us/i/t6textures.jpg/

Soul Calibur 4 uses the "type 8" chunks to define a texture index directly, where Tekken 6 seems to use type 16 and type 20 chunks to actually reference a texture by its global index (as defined by the GIDX chunk in front of the texture data). Here's some code for stumbling through the command lists:

Code: Select all

static int Model_SC4_GetTexFromCommandList(BYTE *cmdBuf, CArrayList<noesisTex_t *> &textures)
{
	BYTE *orig = cmdBuf;
	while (1)
	{
		if (cmdBuf[0] == 8)
		{ //direct texture index
			return GetBigWordRaw(cmdBuf+34);
			//cmdBuf += 56;
		}
		else if (cmdBuf[0] == 18)
		{
			cmdBuf += 32;
		}
		else if (cmdBuf[0] == 16 || cmdBuf[0] == 20)
		{
			int gidx = GetBigWordRaw(cmdBuf+2);
			for (int i = 0; i < textures.Num(); i++)
			{
				noesisTex_t *tex = textures[i];
				if (tex->globalIdx == gidx)
				{
					return i;
				}
			}
			cmdBuf += 24;
		}
		else if (cmdBuf[0] == 0)
		{
			cmdBuf += 4;
		}
		else if (cmdBuf[0] == 63)
		{
			cmdBuf += 4;
		}
		else if (cmdBuf[0] == 65 || cmdBuf[0] == 255)
		{ //seems to indicate the end
			//cmdBuf += 16;
			break;
		}
		else
		{
			break;
		}
	}
	return 0;
}

Does anyone know if these values correspond to any kind of GPU opcodes, or if it's just Namco's own crazy proprietary stuff?

[viperzerofsx]

there are a few jets and helicopter in tekken 6 could i get them out with that program

[revelation]

I'm as clueless as anyone else as far as knowing how to actually get data out of the decompressed bin intelligently, though. Guessing the file tables somewhere in the xex.

As far as i can tell, the file tables are in the IDX files. 8 byte entries. If the file size is invalid (usually 0xFFFFFFFF), skip that entry as it should repeat the offset again.

Code: Select all

uint32 sectorOffset;
uint32 dataSize;

Sector size is 0x8000, so the start offset is sectorOffset * 0x8000. i am guessing the BIN files starting with 0x0FF512ED determine whether a file is compressed or not.

[MrAdults]

Thanks for that revelation. I don't have a fully extracted disc image, so I wasn't aware of the existence of index files. I do happen to own (and love) Tekken 6 for the 360, though, so I'll be extracting a full image to play with in the near future.

And here's some code to handle the new BONE chunk. Everything else short of the material change I've mentioned is 100% identical to Soul Calibur 4.

Code: Select all

typedef struct tkBoneHdr_s
{
	BYTE			id[4];
	int				unknownA;
	int				unknownB; //A&B maybe version?

	WORD			unknownC;
	WORD			unknownD;

	int				numBones;
	float			unknownE;

	int				hieOfs;
	int				transOfs;
	int				rotOfs;
	int				ofs3;
	int				ofs4;
	int				nameOfs;
	int				ofs5;

	int				pad[3];
} tkBoneHdr_t;

//transform a bone
static void Model_SC4_TransformBone(CArrayList<nmdBone_t> &nmdBones, nmdBone_t &bone)
{
	if (bone.didTrans)
	{
		return;
	}
	bone.didTrans = true;
	if (bone.parentIdx >= 0)
	{
		Model_SC4_TransformBone(nmdBones, bone);
	}

	if (bone.parentIdx >= 0)
	{
		Math_MatrixMultiply(&nmdBones[bone.parentIdx].postTransMat, &bone.boneMat, &bone.postTransMat);
	}
	else
	{
		bone.postTransMat = bone.boneMat;
	}
}

//load BONE (used in tekken, also little endian even for 360)
static void Model_SC4_LoadBONE(BYTE *data, CArrayList<nmdBone_t> &nmdBones)
{
	tkBoneHdr_t *hdr = (tkBoneHdr_t *)data;
	for (int i = 0; i < hdr->numBones; i++)
	{
		int *hieData = (int *)(data + hdr->hieOfs);
		nmdBone_t nb;
		memset(&nb, 0, sizeof(nb));
		nb.objName = (char *)(data + hdr->nameOfs + i*32);
		nb.decDat = NULL;
		nb.parentIdx = hieData[i];
		float *pos = (float *)(data + hdr->transOfs + i*12);
		float *ang = (float *)(data + hdr->rotOfs + i*12);
		fourxMatrix_t fm = g_identityMatrix4x4;
		Math_TranslateMatrix4x4(&fm, pos);
		Math_RotateMatrix4x4(&fm, ang[2]*57.295779514f, 0.0f, 0.0f, 1.0f);
		Math_RotateMatrix4x4(&fm, -ang[1]*57.295779514f, 0.0f, 1.0f, 0.0f);
		Math_RotateMatrix4x4(&fm, ang[0]*57.295779514f, 1.0f, 0.0f, 0.0f);
		Math_4x4ToModelMat(&fm, &nb.boneMat);
		nmdBones.Append(nb);
	}

	//transform all of the bones after loading
	for (int i = 0; i < nmdBones.Num(); i++)
	{
		Model_SC4_TransformBone(nmdBones, nmdBones[i]);
	}
}

(contrast this to Soul Calibuer 4, where the bones are matrices in the NMD chunk, and in post-transform space but inverted)

[MrAdults]

Attached is a stupid and blunt tool to extract the idx+bin files, but it requires the bin to be decompressed (if applicable) before-hand. It will name everything with a .raw extension, except files which it can find model data in which will be named with a .nmd extension. (probably not appropriate, but it's the extension I started using for this format due to SC4)

I've still only looked through the 301 bin to any extent so far, but I've found that most of the characters' parts and crap are not in the same files:
http://img36.imageshack.us/i/alisay.jpg/
My model loader handles infinite layers of data for the stacked containers (most of the files are stacked), but that doesn't reveal a lot. Most commonly it just produces some extra texture sets (they like to store head textures in separate chunks for some reason).

I'm curious if there are some more complete "decked out" character models, maybe for cinematics or other canned purposes, in the other .bin files. Otherwise putting them together could be a real pain, unless someone finds the data that links all of that stuff together.

Edit: Got around to checking out the other bins, turns out almost all of the default costumes are in single packages in some of the later bins.

And http://img525.imageshack.us/img525/3109/t6chars.jpg

[ALYX]

What kind of tool can open *nmd file ?

[d2rnattakorn]

Wow, finally tekken 6 is extracted able
I am Thank MrAdulte for the BIN extractor.

I am just wondering on something after reading through the post.
If the format is identical to Soul Calibur 4, would we be getting .SPD files after BIN extraction and if it is this process would be easy to make the extraction with the SPD_EX tool?