[REL/REQ] Sleeping Dogs Part 2: Definitive Edition

[SergeantJoe]

Alright, hello!

I bet you're all going "oh god, not this again", but hold on, just give me a minute.

As you may know Sleeping Dogs Definitive Edition was released yesterday. It's basically the same game, only with MASSIVELY upgraded visuals, sounds, animations, and with several gameplay changes.

There have already been quite a few tools made for the original Sleeping Dogs, but they only worked half the time, and now they don't work at all. Let me break down all the parts using all the info I got:


ARCHIVES

Sleeping Dogs uses an engine that was custom-built by UFG specifically for this series. All files are compressed into .PERM.BIN/.TEMP.BIN files, which are packed into .BIG archives with a .BIX header.

Ekey created an unpacker, and was nice enough to quickly update it for Definitive Edition compatibility.

When you unpack a .BIG file, you'll end up with "../SleepingDogsDefinitiveEdition/*NameOfBig*/Data/". If you move the Data folder into the main game folder and remove the BIG file, the game will load the unpacked files directly.

There are some issues with this though. The unpacker doesn't assign filenames to everything, so when the games tries to load the unnamed files it crashes.


SOUNDS

All sounds are stored in Audiokinetc Wwise .PCK and .BNK files. These can be extracted with Ravioli Game Tools.


MESHES

Model vertices, UVs, and weights are stored inside .PERM.BIN files. Marcius Szkaradek created a Blender script that supports all of those, but it has some issues. Full compatibility list:

Characters
Intact mesh, Full UVs, Weights

Vehicles
Slightly scaled mesh, Full UVs, Weights (decal UVs must be obtained with this modification. Tires have no UVs at all.)

Props/Weapons
Intact mesh, UVs mixed. Some have UVs, some do not.

Buildings
Intact mesh, No UVs.


TEXTURES

.PERM.BIN files can also store textures. The .PERM.BIN file contains the names, and an accompanying .TEMP.BIN file contains the actual image.

I managed to create an extractor, but some but be extracted by hand. See below.




BONES

This game uses the Havok Engine, and all the skeletons are stored in one file. Only Shakotay knows how to extract them.


ANIMATIONS

The animations are in Havok format as well. I've had five people offer to help with these, then promptly get bored and quit or simply ignore me.

Also, interesting to note they they now use version 2013 instead of the previous 2011.




Most of the work is already done. All that's left is to just make the tools work with these new files.

The good news is, I've looked through files from all of the categories, and they seemed practically identical to the old ones. Only difference is a different header.


Forum rules prohibit posting links in public, so PM me for file samples.

[Theuaredead`]

Well apparently you don't have to pay cash to get in anymore, so I'll post regarding the TEMP.BINs and PERM.BINs
The textures above are not done by a hacky method, I used Intel GPA to rip those.

These I got out of a hacky method.





It appears that the TEMP.BINs are compressed the same way as the original Sleeping Dogs, it just seems the code to the PERM.BINs are slightly different so TEXRIPPER can't read them to extract the data.

[cra0]

Well apparently you don't have to pay cash to get in anymore, so I'll post regarding the TEMP.BINs and PERM.BINs
The textures above are not done by a hacky method, I used Intel GPA to rip those.

These I got out of a hacky method.

why would you even..

just open up the character bins they got textures

[SergeantJoe]

TextureFinder works, but only for the diffuse and specular maps. The normal maps are broken no matter what settings are used.

Also it's worth mentioning they changed the format of the normal maps. They are now just like regular normal maps, only with an inverted blue channel.

[cra0]

TextureFinder works, but only for the diffuse and specular maps, the normal maps are broken no matter what settings are used.

they aren't broken msg me a sample ill get it working

[Theuaredead`]

Do you want a normal exported VIA texture finder?

why would you even..

just open up the character bins they got textures

because its a pain in the ass to splice 2048x2048 and 1024x1024 textures back together with texture finder.
Plus I wanted to see how much of the format was the same.

[cra0]

there easy

ATI2N now i sleep

[Theuaredead`]

there easy

ATI2N now i sleep

now its forever a mystery!
assuming sergeant joe didnt send cra0 a normal map from Intel GPA just to invert the blue channel
still gonna assume that there's something incorrect with the DXT Header for the normal maps.

oh i guess there's no spoiler tags

edit:
woah, getting some deja vu, alpha channel has apart of normal map. Sadly, I already know that the format probably doesn't do that by default, Intel GPA exports a Source Engine like normal with a inverted blue channel.

[SergeantJoe]

I just sent him a .temp.bin file.

Also, apparently ATI2N is a type of compression format. I don't know of any programs that support it, so we'll just have to wait until he gets back.

[SergeantJoe]

As noted, the executable's PDB file was included in the game files, and after a conversion to human-readable format courtesy of cra0, I managed to dig up some interesting stuff.

Code: Select all

/* 2853 */
enum hkxVertexDescription::DataUsage
{
  HKX_DU_NONE = 0x0,
  HKX_DU_POSITION = 0x1,
  HKX_DU_COLOR = 0x2,
  HKX_DU_NORMAL = 0x4,
  HKX_DU_TANGENT = 0x8,
  HKX_DU_BINORMAL = 0x10,
  HKX_DU_TEXCOORD = 0x20,
  HKX_DU_BLENDWEIGHTS = 0x40,
  HKX_DU_BLENDINDICES = 0x80,
  HKX_DU_USERDATA = 0x100,
};

/* 2854 */
enum hkxMaterial::TextureType
{
  TEX_UNKNOWN = 0x0,
  TEX_DIFFUSE = 0x1,
  TEX_REFLECTION = 0x2,
  TEX_BUMP = 0x3,
  TEX_NORMAL = 0x4,
  TEX_DISPLACEMENT = 0x5,
  TEX_SPECULAR = 0x6,
  TEX_SPECULARANDGLOSS = 0x7,
  TEX_OPACITY = 0x8,
  TEX_EMISSIVE = 0x9,
  TEX_REFRACTION = 0xA,
  TEX_GLOSS = 0xB,
  TEX_DOMINANTS = 0xC,
  TEX_NOTEXPORTED = 0xD,
};

/* 2856 */
enum hkxMaterial::UVMappingAlgorithm
{
  UVMA_SRT = 0x0,
  UVMA_TRS = 0x1,
  UVMA_3DSMAX_STYLE = 0x2,
  UVMA_MAYA_STYLE = 0x3,
};

/* 2859 */
enum hkxVertexDescription::DataType
{
  HKX_DT_NONE = 0x0,
  HKX_DT_UINT8 = 0x1,
  HKX_DT_INT16 = 0x2,
  HKX_DT_UINT32 = 0x3,
  HKX_DT_FLOAT = 0x4,
};

/* 2862 */
enum hkxIndexBuffer::IndexType
{
  INDEX_TYPE_INVALID = 0x0,
  INDEX_TYPE_TRI_LIST = 0x1,
  INDEX_TYPE_TRI_STRIP = 0x2,
  INDEX_TYPE_TRI_FAN = 0x3,
  INDEX_TYPE_MAX_ID = 0x4,
};

/* 2863 */
enum hkxVertexDescription::DataHint
{
  HKX_DH_NONE = 0x0,
  HKX_DH_LIGHTMAP = 0x1,
};

/* 2867 */
enum hkxMaterial::PropertyKey
{
  PROPERTY_MTL_TYPE_BLEND = 0x1,
  PROPERTY_MTL_UV_ID_STAGE0 = 0x100,
  PROPERTY_MTL_UV_ID_STAGE1 = 0x101,
  PROPERTY_MTL_UV_ID_STAGE2 = 0x102,
  PROPERTY_MTL_UV_ID_STAGE3 = 0x103,
  PROPERTY_MTL_UV_ID_STAGE4 = 0x104,
  PROPERTY_MTL_UV_ID_STAGE5 = 0x105,
  PROPERTY_MTL_UV_ID_STAGE6 = 0x106,
  PROPERTY_MTL_UV_ID_STAGE7 = 0x107,
  PROPERTY_MTL_UV_ID_STAGE8 = 0x108,
  PROPERTY_MTL_UV_ID_STAGE9 = 0x109,
  PROPERTY_MTL_UV_ID_STAGE10 = 0x10A,
  PROPERTY_MTL_UV_ID_STAGE11 = 0x10B,
  PROPERTY_MTL_UV_ID_STAGE12 = 0x10C,
  PROPERTY_MTL_UV_ID_STAGE13 = 0x10D,
  PROPERTY_MTL_UV_ID_STAGE14 = 0x10E,
  PROPERTY_MTL_UV_ID_STAGE15 = 0x10F,
  PROPERTY_MTL_UV_ID_STAGE_MAX = 0x110,
};

/* 2868 */
enum hkxMaterial::Transparency
{
  transp_none = 0x0,
  transp_alpha = 0x2,
  transp_additive = 0x3,
  transp_colorkey = 0x4,
  transp_subtractive = 0x9,
};

No idea how useful any of this is, but I thought it may be helpful.

[SergeantJoe]

cra0 finally explained how to extract textures!

In order to extract textures from Sleeping Dogs Definitive Edition we basically have to cut the texture bytes out of the .TEMP.BIN file and glue on a proper .DDS header.
It's a bit of a long and obnoxious process though, so bear with me.

You'll need:
- TextureFinder
- A hex editor (I'll be using TinyHexer since it's free and readily available)
- WTV


1. Open the .temp.bin in TextureFinder and align the texture you want to the top-left corner.



Now, note the Width, Quad Format, and Offset.

For example, this is a 1024x1024 DXT5 texture that starts at offset 1398112.

2. Now that we know where it starts, we need to remove all the extra junk around it.

Open the .temp.bin in a hex editor and go to that offset.




That "ÿ" is where the texture starts. Now, delete everything before that point.




3. Once we've isolated the texture bytes, we now need to attach them to a valid DDS header. This is literally as simple as doing Select All, Ctrl-C, Ctrl-V.




Now the tricky part. The information in the header needs to match the format of the texture. There are three importants bits:

Width


Height


Pitch



There is a lot of incredibly complicated math involved here, so here's a quick cheat sheet for the most common types/sizes.

Code: Select all

----------512x512----------
Width/Height       = 02
Pitch (DXT1)       = 02
Pitch (DXT5/ATI2N) = 04

---------1024x1024---------
Width/Height       = 04
Pitch (DXT1)       = 08
Pitch (DXT5/ATI2N) = 10

---------2048x2048---------
Width/Height       = 08
Pitch (DXT1)       = 20
Pitch (DXT5/ATI2N) = 40

Make sure the header Width, Height, Pitch, and DXT format match those of the texture.

4. Once everything matches, save the file as .DDS, and you're done!




5. However, this is just the method for diffuse and specular maps.

Set TextureFinder to DXT5, then look for something like this:




This is the normal map. Note the height and offset, then follow the same process, only using the ATI2N header template in the link above.

Pitch calculation is identical to DXT5, so 02 for 512x512 and 10 for 1024x1024.

Only WTV is capable of previewing ATI2N files as far as I know.

[SergeantJoe]

Also, as it turns out, .TEMP.BIN files are just a bunch of plain .DDS files glued together!

Using a slight modification of the above method, it's finally possible to make texture mods for the game!





But only for cutscenes, not the actual gameplay itself.

[Theuaredead`]

I feel stupid for not realizing those were DDS data

edit:

had some fun splicing in my beta Wei Shen remake

[SergeantJoe]

Quick update on the hex values list:

Code: Select all

----------512x512----------
Width/Height       = 02
Pitch (DXT1)       = 02
Pitch (DXT5/ATI2N) = 04

---------1024x1024---------
Width/Height       = 04
Pitch (DXT1)       = 08
Pitch (DXT5/ATI2N) = 10

---------2048x2048---------
Width/Height       = 08
Pitch (DXT1)       = 20
Pitch (DXT5/ATI2N) = 40

And here's the forumla itself:

Code: Select all

int pitch = ((width + 3) / 4) * ((height + 3) / 4) * ((32 + 3) / 4);

if (compressionFormat == 1)
{
	pitch *= 1; // Multiply by 1 for DXT1
}
else if (compressionFormat == 5)
{
	pitch *= 2; // Multiply by 2 for DXT5/ATI2N
}

This will give you the pitch value, but then you need to convert it to hex. I use a DDS script for 010 Editor.

[Theuaredead`]

Appears I found out the resolution header.

The header is about 3 lines after the mention of the name of the texture in the PERM.BIN


That's a 2048x2048 texture with the correct header from the DDS headers,


Same here, but for the 1024x1024

Now I gotta figure out what tells it to be DXT1 and DXT5