READ THE RULES: Click here

Follow us on Facebook: https://www.facebook.com/xentax/ :)

Researching of a .dat archive with a .mnf file.

The Original Forum. Game archives, full of resources. How to open them? Get help here.
Lugnut360
ultra-n00b
Posts: 1
Joined: Fri Aug 16, 2013 2:47 pm

Re: Researching of a .dat archive with a .mnf file.

Post by Lugnut360 » Tue Aug 20, 2013 2:29 am

Do you unpack the files from the .dat, then use a zlib decompression on them or do you somehow run zlib decompression on the entire eso0000.dat?

I'm very new to computer programming/datamining and would appreciate any explanation of how you write the extraction script for this.

When I open up eso0000.dat in hex editor, I see:
Hex: ASCII in parenthesis:
50 45 53 32 (PES2)
01 00 00 00 (....)
00 00 0E 00 (....)
00 00 78 9C (x..}) - zlib file header

I'm not sure how you write a BMS script from this. I assume something like:

idstring "PES2"
get UNK1
get FILES long
for i = 0 < FILES
getdstring NAME (????)
Not sure how to find OFFSET, ZSIZE, SIZE

Thanks TERAB1T for the unpacked file!

Droolie
veteran
Posts: 114
Joined: Fri Aug 19, 2005 4:31 pm
Has thanked: 14 times
Been thanked: 33 times

Re: Researching of a .dat archive with a .mnf file.

Post by Droolie » Tue Aug 20, 2013 10:02 am

So does anyone know yet what's in the 21 unknown bytes at the beginning and the data after the 3rd zlib block of eso.mnf? Or what data the first two blocks contain? I'm pretty sure they'd have to at least keep references to the filetables there...
And I'm sure you have this already by now, but for those who don't, this QuickBMS script unpacks the 3 zlibbed blocks we know of from a *.mnf file:
*snip - you can find the script a few posts after this*
I would've used a loop but I'm planning to expand the script to get the data from the blocks too, that's why I always added the file number. ;)

EDIT -> Block 2 seems to contain data about the files as well: for every file just one field of 8 bytes, little endian. I have no idea what block 1 is.
Last edited by Droolie on Fri Oct 04, 2013 1:13 pm, edited 1 time in total.

Ekey
M-M-M-Monster veteran
M-M-M-Monster veteran
Posts: 1638
Joined: Wed Mar 31, 2010 6:54 am
Has thanked: 62 times
Been thanked: 821 times

Re: Researching of a .dat archive with a .mnf file.

Post by Ekey » Tue Aug 20, 2013 10:46 am

The third block from MNF is the file table.

Code: Select all

0x4 - szFileSize
0x4 - szFileZSize
0x4 - szUnknown01
0x4 - szOffset
0x1 - szComType(0 - Not Compressed, 1 - Zlib, 2 - Snappy)
0x1 - szArchiveNum
0x2 - szUnknown02
Hash Function: Initialize hash by 0xA8396u

http://rdf3x.googlecode.com/svn/trunk/i ... l/Hash.cpp
http://rdf3x.googlecode.com/svn/trunk/i ... l/Hash.hpp
Last edited by Ekey on Tue Aug 20, 2013 11:16 am, edited 1 time in total.

Droolie
veteran
Posts: 114
Joined: Fri Aug 19, 2005 4:31 pm
Has thanked: 14 times
Been thanked: 33 times

Re: Researching of a .dat archive with a .mnf file.

Post by Droolie » Tue Aug 20, 2013 11:15 am

Exactly, but it might not be linked with the filenames. What if block 1 or 2 (or the extra data after block 3) contains that link and block 3 just contains sizes and offsets?

Ekey
M-M-M-Monster veteran
M-M-M-Monster veteran
Posts: 1638
Joined: Wed Mar 31, 2010 6:54 am
Has thanked: 62 times
Been thanked: 821 times

Re: Researching of a .dat archive with a .mnf file.

Post by Ekey » Tue Aug 20, 2013 11:18 am

In 3 block - 0x4 - szUnknown01 - it's seems filename hash (see my previous post)

Droolie
veteran
Posts: 114
Joined: Fri Aug 19, 2005 4:31 pm
Has thanked: 14 times
Been thanked: 33 times

Re: Researching of a .dat archive with a .mnf file.

Post by Droolie » Tue Aug 20, 2013 1:51 pm

Yes, that will help a lot once we have found the complete list of filenames, but we still need to find that, right?

By the way, these are the 15 unknown bytes after the MES2 header:

Code: Select all

             unknown1  num_dat_archives       EOF_offset           dummy         unknown2
FILENAME         [02 00] [XX] 00 00 00 00 [XX XX XX XX] 00 03 [00 00 00] 04 [XX XX XX XX]
eso.mnf:         [02 00] [D0] 00 00 00 00 [7B FB 4F 00] 00 03 [00 00 00] 04 [00 08 0C 01]
esoaudioen.mnf:  [02 00] [04] 00 00 00 00 [2B 09 1E 00] 00 03 [00 00 00] 04 [00 04 06 01]
game.mnf:        [02 00] [01] 00 00 00 00 [3A A2 00 00] 00 03 [00 00 00] 04 [00 00 10 19]

unknown1: This always has to be 2 for some reason. The files don't open otherwise.
num_dat_archives: number of DAT files
EOF_offset (little endian): After reading these 4 bytes, add EOF_offset to current offset to reach the end of the third block. This is EOF except for eso.mnf which has extra data.
unknown2 (big endian): This is 4120*(2^n)+1, with n ranging from 0 to 2. It seems n is also the number of the DAT archive containing the ZOSFT, but maybe this is just a coincidence.
EDIT -> I'm in the process of decompiling eso.exe. It's not going that well as expected but I fixed a few things in the description above.

Droolie
veteran
Posts: 114
Joined: Fri Aug 19, 2005 4:31 pm
Has thanked: 14 times
Been thanked: 33 times

Re: Researching of a .dat archive with a .mnf file.

Post by Droolie » Fri Oct 04, 2013 10:31 am

Bumping this topic.
About a month and a half has passed. I got stuck in the decompilation process.
Has anybody made any progress?

Here are my scripts so far. My code is a bit messy but then again, QuickBMS isn't that clean as a language. TERAB1T, it would be great if you could share how you linked the ZOSFT filenames to the files. :)

TERAB1T
n00b
Posts: 13
Joined: Mon May 28, 2012 12:40 am
Has thanked: 1 time
Been thanked: 5 times

Re: Researching of a .dat archive with a .mnf file.

Post by TERAB1T » Sat Oct 12, 2013 2:34 am

About game.mnf filenames:

There are three zlib-ed blocks in .mnf file. As you know, the last one contains filesizes, offsets etc (20 bytes for each file). And the second one contains indexes for some of filenames from zosft-file: 8 bytes for each file. Look at this screenshot:

Image

If 'Unknown1 = 0' and 'Unknown3 = 128 (decimal)' index (Unknown0) will be valid.
And 'Unknown1 = 1' and 'Unknown3 = 128' is filetable (zosft).

Use valid indexes and link them with files at the end of filetable (after 9 zlibed blocks).

Also I figured out, that if you remove all the "00 00 00 00" parts from the first block in .mnf, you'll get unique ID for each file in archives (4 bytes for each file).

Droolie
veteran
Posts: 114
Joined: Fri Aug 19, 2005 4:31 pm
Has thanked: 14 times
Been thanked: 33 times

Re: Researching of a .dat archive with a .mnf file.

Post by Droolie » Sun Oct 13, 2013 12:47 pm

Interesting - there are only 1768 valid indices in the MNF, although there are 1772 records in the ZOSFT file. It's not so hard to combine those indices with the ZOSFT without checking their validity though: they return in ZOSFT block 2.3 (that is, the third zlib block of the second big block, so it's the 6th zlib block), along with offsets for the corresponding filenames.
The indices are also unique in game.mnf, while they aren't in eso.mnf... quite strange indeed. I think that when combined with another number (which isn't necessary for game.mnf), they might be unique.

I had also found out that there are unique IDs for the files in the first MNF block (check the ZOSFT script in my previous post, it used them), and these IDs appear again in a few of the ZOSFT blocks, but they seem to be in the wrong order. The indices in block 2 seem to be the only link. I called that index file_number in my research notes which I made available here: http://pastebin.com/rsgh1irr

I adapted my scripts so the filenames of game.zosft are correctly applied. :)

Luriam
beginner
Posts: 38
Joined: Mon Jun 25, 2012 8:44 am
Has thanked: 47 times

Re: Researching of a .dat archive with a .mnf file.

Post by Luriam » Sun Nov 24, 2013 10:00 am

Hey guys, anyone succeeded at extracting the sounds ? eso0000.dat through eso0207.dat should contain sfx. And esoaudioen0000.dat through esoaudioen0003.dat got to be the container for the voices.

I have attached a sample of the voice archive HERE. (1,09 gb)
And also the esoaudioen.mnf, just in case.

And the possible sfx HERE.

Anyone gets anything, let me know. Thanks!

Droolie
veteran
Posts: 114
Joined: Fri Aug 19, 2005 4:31 pm
Has thanked: 14 times
Been thanked: 33 times

Re: Researching of a .dat archive with a .mnf file.

Post by Droolie » Wed Nov 27, 2013 8:16 pm

Extracting the audio is easy, you can do it with my QuickBMS scripts to extract everything using the MNF files. You'd better just extract all the archives in one go though, because the files the MNF file points to is divided over all of those DAT files.
Psst! If you know how to use Google, you can also find the soundtrack which I ripped and posted online some time ago...

stalja
beginner
Posts: 32
Joined: Sat Sep 12, 2009 11:33 am
Has thanked: 10 times
Been thanked: 5 times

Re: Researching of a .dat archive with a .mnf file.

Post by stalja » Thu Jan 09, 2014 4:33 pm

I believe eso.mnf header has changed. Looking at this
Droolie wrote:             unknown1  num_dat_archives       EOF_offset           dummy         unknown2
FILENAME         [02 00] [XX] 00 00 00 00 [XX XX XX XX] 00 03 [00 00 00] 04 [XX XX XX XX]
eso.mnf:         [02 00] [D0] 00 00 00 00 [7B FB 4F 00] 00 03 [00 00 00] 04 [00 08 0C 01]
esoaudioen.mnf:  [02 00] [04] 00 00 00 00 [2B 09 1E 00] 00 03 [00 00 00] 04 [00 04 06 01]
game.mnf:        [02 00] [01] 00 00 00 00 [3A A2 00 00] 00 03 [00 00 00] 04 [00 00 10 19]
I do not think bytes are in this order anymore. (there are extra bytes now and some are not all 00)

stalja
beginner
Posts: 32
Joined: Sat Sep 12, 2009 11:33 am
Has thanked: 10 times
Been thanked: 5 times

Re: Researching of a .dat archive with a .mnf file.

Post by stalja » Fri Jan 10, 2014 12:49 pm

It seems that that unknown block of data at the end of eso.mnf you mentioned previously is now inserted after the first 22 bytes. I'll try and create a functioning bms script to compensate for this, but I'm a noob so ...

stalja
beginner
Posts: 32
Joined: Sat Sep 12, 2009 11:33 am
Has thanked: 10 times
Been thanked: 5 times

Re: Researching of a .dat archive with a .mnf file.

Post by stalja » Fri Jan 10, 2014 4:26 pm

For the latest build things seem to have changed in eso.mnf. Block of dat that used to be after the third zlib block is now moved to 22nd byte. I modified Droolie's quickbms script to compensate for this change. I might have broken something else, but I did manage to extract a lot of DDS textures and gr2 models this way. WAVs seem to be broken (won't play in media players), but it might be something else.

There does not seem to be a ZOSFT file extracted with this script from eso*.dat files so I could not use the 2nd Droolie script to connect filenames with files. Any help is appreciated.
You do not have the required permissions to view the files attached to this post.

fxfire
ultra-n00b
Posts: 1
Joined: Sun Mar 25, 2012 4:34 pm

Re: Researching of a .dat archive with a .mnf file.

Post by fxfire » Fri Jan 10, 2014 7:52 pm

Thanks for all the great work so far. Does anyone have a hint what type of packer/obfuscator was used for the current stress test beta exo.exe ?

Post Reply