The Division SDF Archive Format

[Gh0stBlade]

Hi

Not a programmer, to be onoest im more of a interested hobbyist on modding. i can parse out all the files using the rouge extractor but how do i find the art, textures, all i find is the mphys files and mmb files. I dont know how to find the art files like DDS or the 3D files.

i really do appreciate the knowledge of people here. Hopefully someone can give me a poke to right direction


//cheers

Textures should all be plain dds files. Most art assets are located in rogue/baked/art. ".mmb" are the 3D model files but they're not in a native format like textures.

Regards.

[kaffeblask]

did the trick - i didnt decompile the whole project - only parts. thanks for giving me the pointer!
Appreciated!

/cheers!

[kaffeblask]

Since the DDS format is BC7 compression, i found a plugin Intel has made that works for latest Photoshop CC versions.
This can read the file into photoshop.

link: http://gametechdev.github.io/Intel-Text ... ks-Plugin/
direct: https://github.com/GameTechDev/ITW-Beta ... _1.0.4.zip


//cheers

[w00t]

Compressed data block represents binary search tree (for strings), with files in leaves.

Could you provide any info on the binary search tree structure? I'm able to discern the file names but I'm unable to determine how the data fits together.

My breakdown so far:

Code: Select all

	
    0x72, 0xC4, 0x9F, 0x01, 0x00, 0x6E, 0x04, 0xA8, 0x00, 0x00, 0x6D, 0x7D,
    0x9E, 0x00, 0x00, 0x66, 0x28, 0x01, 0x00, 0x00, 0x63, 0x9B, 

    // -- Should bypass this header? if it is a header that is --
    
    0x00, 0x00, 0x00, 0x01, // int32 string length (0x01, 1)
    0x62, // string ( 1 byte, 'b')

    0x6C, 0x77, 0x00, 0x00, 0x00, // unknown

    0x0E, // int32 string length (0x0E, 14)
    0x61, 0x64, 0x5F, 0x77, 0x6F, 0x72, 0x64, 0x73, 0x2F, 0x63, 0x68, 0x61, 0x74, 0x5F, // string (14 bytes, 'ad_words/chat_')
    
    0x66, 0x55, 0x00, 0x00, 0x00, // unknown

    0x0E, // int8, string length (0x0E, 14)
    0x65, 0x6E, 0x5F, 0x62, 0x61, 0x64, 0x77, 0x6F, 0x72, 0x64, 0x2E, 0x63, 0x73, 0x76, // string (14 bytes, 'en_badword.csv')

    0x42, 0xA3, 0x18, 0xBA, 0x1E, 0x00, 0x25, // unknown 
    0x72, 0x0E, // int16, file length of 'badwords/chat_en_badword.csv' which is 3698
            
    0x3F, 0x09, 0x38, 0x01, 0x00, 0xD1, 0xC6, 0x76, 0xF4,  // unknown

    0x0E, // int8, string length, 14 decimal
    0x66, 0x72, 0x5F, 0x62, 0x61, 0x64, 0x77, 0x6F, 0x72, 0x64, 0x2E, 0x63, 0x73, 0x76, // string (14 bytes, 'fr_badword.csv')
    
    0x42, 0xA3, 0x18, 0xBA, 0x1E, 0x00, 0x29, // unknown
    0x5F, 0x0A // int16, file length of 'badwords/chat_fr_badword.csv' which is 2655

The combination of those three strings gives the path 'badwords/chat_en_badword.csv' which is valid, but I just do not know what the correct structure should be to read these entries out correctly.

Any help would be appreciated, thanks.

*edited to include the file sizes which I've found

[xastey]

EDIT: and how to use your tool?
EDIT2: nvm, know how to extract
EDIT3: Maybe someone is able to rebuild the ingame map. A lot of ppl would love to get hand on it since all available are stitched screenshots

-> Open CMD
-> cd to roug_sdf.exe
-> type rouge_sdf.exe <.sdftoc path> <output directory>
-> Press Enter
-> Done

Thanks man, needed to extra some assets for an web app i'm building.. this worked perfectly!

[lunger]

Since the DDS format is BC7 compression, i found a plugin Intel has made that works for latest Photoshop CC versions.
This can read the file into photoshop.

link: http://gametechdev.github.io/Intel-Text ... ks-Plugin/
direct: https://github.com/GameTechDev/ITW-Beta ... _1.0.4.zip


//cheers

Some of the textures are DX10, you can open them with VS2015 CE.

[cameleot]

Is there a way to open/convert the .mmb model files to .obj or such?

[skd23]

Hello, I'm not really a programmer. But is there a way to change Russian files with English? I'm desperate to change to English.

[Sefer]

For converting the sound files... Seems like some cannot be extracted and some are broken... Atleast you can use some

viewtopic.php?p=66311&sid=94de722e28428 ... e2f#p66311

[eibw3n]

Compressed data block represents binary search tree (for strings), with files in leaves.

Could you provide any info on the binary search tree structure? I'm able to discern the file names but I'm unable to determine how the data fits together.

My breakdown so far:

Code: Select all

	
    0x72, 0xC4, 0x9F, 0x01, 0x00, 0x6E, 0x04, 0xA8, 0x00, 0x00, 0x6D, 0x7D,
    0x9E, 0x00, 0x00, 0x66, 0x28, 0x01, 0x00, 0x00, 0x63, 0x9B, 

    // -- Should bypass this header? if it is a header that is --
    
    0x00, 0x00, 0x00, 0x01, // int32 string length (0x01, 1)
    0x62, // string ( 1 byte, 'b')

    0x6C, 0x77, 0x00, 0x00, 0x00, // unknown

    0x0E, // int32 string length (0x0E, 14)
    0x61, 0x64, 0x5F, 0x77, 0x6F, 0x72, 0x64, 0x73, 0x2F, 0x63, 0x68, 0x61, 0x74, 0x5F, // string (14 bytes, 'ad_words/chat_')
    
    0x66, 0x55, 0x00, 0x00, 0x00, // unknown

    0x0E, // int8, string length (0x0E, 14)
    0x65, 0x6E, 0x5F, 0x62, 0x61, 0x64, 0x77, 0x6F, 0x72, 0x64, 0x2E, 0x63, 0x73, 0x76, // string (14 bytes, 'en_badword.csv')

    0x42, 0xA3, 0x18, 0xBA, 0x1E, 0x00, 0x25, // unknown 
    0x72, 0x0E, // int16, file length of 'badwords/chat_en_badword.csv' which is 3698
            
    0x3F, 0x09, 0x38, 0x01, 0x00, 0xD1, 0xC6, 0x76, 0xF4,  // unknown

    0x0E, // int8, string length, 14 decimal
    0x66, 0x72, 0x5F, 0x62, 0x61, 0x64, 0x77, 0x6F, 0x72, 0x64, 0x2E, 0x63, 0x73, 0x76, // string (14 bytes, 'fr_badword.csv')
    
    0x42, 0xA3, 0x18, 0xBA, 0x1E, 0x00, 0x29, // unknown
    0x5F, 0x0A // int16, file length of 'badwords/chat_fr_badword.csv' which is 2655

The combination of those three strings gives the path 'badwords/chat_en_badword.csv' which is valid, but I just do not know what the correct structure should be to read these entries out correctly.

Any help would be appreciated, thanks.

*edited to include the file sizes which I've found

Did you make any more progress on this? I am basically stuck at the same place at the moment.
My assumption is that those unknown blocks in your post must somehow map to file id (a,b,c / 1-2799), offset inside that file and the a length. You also had the file length at 2 bytes which would only work for 64k file sizes which would be a bit small.

Any help would be appreciated.

[w00t]

Did you make any more progress on this? I am basically stuck at the same place at the moment.
My assumption is that those unknown blocks in your post must somehow map to file id (a,b,c / 1-2799), offset inside that file and the a length. You also had the file length at 2 bytes which would only work for 64k file sizes which would be a bit small.

Any help would be appreciated.

I haven't gotten any further with it. I've tried finding relationships between the data in the unknown blocks or even just a pattern with how they are structured, but no luck.

You're right that 64k would be too small, especially for textures and audio, but I highlighted what I found because they happened to match up exactly with the sample data. I guess it's possible the byte prior to the file length indicates a 'compression' level of an in32 or 64, so that they don't waste bytes on files which are <64k. I don't know if that's realistic though. I'd need to find a file in the data which was large and test the theory.

[eibw3n]

I haven't gotten any further with it. I've tried finding relationships between the data in the unknown blocks or even just a pattern with how they are structured, but no luck.

You're right that 64k would be too small, especially for textures and audio, but I highlighted what I found because they happened to match up exactly with the sample data. I guess it's possible the byte prior to the file length indicates a 'compression' level of an in32 or 64, so that they don't waste bytes on files which are <64k. I don't know if that's realistic though. I'd need to find a file in the data which was large and test the theory.

Yep, this could be it.

I'd love if m0xf could release the source for his rogue_sdf.exe since I usually don't trust random executables I find on the internet
Alternatively if no source, m0xf could chime in and give some more information on the sdftoc format.

[Sir Kane]

Or you could challenge yourself and try to figure it out on your own!

[eibw3n]

Or you could challenge yourself and try to figure it out on your own!

I fully agree with this. I like a challenge. However I've been stuck at this for a while now and I am starting to think I am missing / not seeing something very basic. That's why I wouldn't mind a pointer in the right direction.

This is not the first such pack file that I want to try to write my own unpacker for but it's the first one that gives me problems. Most I've looked at are very pretty straight forward. They usually have a toc with filenames, at offset and lengths with a bit of compression and special headers mixed in. I haven't seen anything like the format that Division uses before.

[w00t]

I haven't gotten any further with it. I've tried finding relationships between the data in the unknown blocks or even just a pattern with how they are structured, but no luck.

You're right that 64k would be too small, especially for textures and audio, but I highlighted what I found because they happened to match up exactly with the sample data. I guess it's possible the byte prior to the file length indicates a 'compression' level of an in32 or 64, so that they don't waste bytes on files which are <64k. I don't know if that's realistic though. I'd need to find a file in the data which was large and test the theory.

Yep, this could be it.

I'd love if m0xf could release the source for his rogue_sdf.exe since I usually don't trust random executables I find on the internet
Alternatively if no source, m0xf could chime in and give some more information on the sdftoc format.

I've done a bit more poking around and my theory about file sizes falls apart with >64k files. I can't find their sizes anywhere so I'm assuming they are not the file sizes but maybe some kind of offset in the data files for that entry.

It would make sense that the entry after 'chat_en_badword.csv' started at the offset which was the file size of 'chat_en_badword.csv', for example.