XeNTaX Forum Index
Forum MultiEx Commander Tools Tools Home
It is currently Tue Oct 17, 2017 4:25 pm

All times are UTC + 1 hour


Forum rules


Please click here to view the forum rules



Post new topic Reply to topic  [ 63 posts ]  Go to page Previous  1, 2, 3, 4, 5  Next
Author Message
 Post subject: Re: Researching of a .dat archive with a .mnf file.
PostPosted: Tue Aug 20, 2013 2:29 am 
Offline
ultra-n00b

Joined: Fri Aug 16, 2013 2:47 pm
Posts: 1
Has thanked: 0 time
Have thanks: 0 time














You can make the ads go away by registering

Do you unpack the files from the .dat, then use a zlib decompression on them or do you somehow run zlib decompression on the entire eso0000.dat?

I'm very new to computer programming/datamining and would appreciate any explanation of how you write the extraction script for this.

When I open up eso0000.dat in hex editor, I see:
Hex: ASCII in parenthesis:
50 45 53 32 (PES2)
01 00 00 00 (....)
00 00 0E 00 (....)
00 00 78 9C (x..}) - zlib file header

I'm not sure how you write a BMS script from this. I assume something like:

idstring "PES2"
get UNK1
get FILES long
for i = 0 < FILES
getdstring NAME (????)
Not sure how to find OFFSET, ZSIZE, SIZE

Thanks TERAB1T for the unpacked file!


Top
 Profile  
 
 Post subject: Re: Researching of a .dat archive with a .mnf file.
PostPosted: Tue Aug 20, 2013 10:02 am 
Offline
veteran

Joined: Fri Aug 19, 2005 4:31 pm
Posts: 112
Has thanked: 14 times
Have thanks: 32 times
So does anyone know yet what's in the 21 unknown bytes at the beginning and the data after the 3rd zlib block of eso.mnf? Or what data the first two blocks contain? I'm pretty sure they'd have to at least keep references to the filetables there...
And I'm sure you have this already by now, but for those who don't, this QuickBMS script unpacks the 3 zlibbed blocks we know of from a *.mnf file:
*snip - you can find the script a few posts after this*
I would've used a loop but I'm planning to expand the script to get the data from the blocks too, that's why I always added the file number. ;)

EDIT -> Block 2 seems to contain data about the files as well: for every file just one field of 8 bytes, little endian. I have no idea what block 1 is.


Last edited by Droolie on Fri Oct 04, 2013 1:13 pm, edited 1 time in total.

Top
 Profile  
 
 Post subject: Re: Researching of a .dat archive with a .mnf file.
PostPosted: Tue Aug 20, 2013 10:46 am 
Offline
M-M-M-Monster veteran
M-M-M-Monster veteran

Joined: Wed Mar 31, 2010 6:54 am
Posts: 1593
Has thanked: 59 times
Have thanks: 748 times
The third block from MNF is the file table.

Code:
0x4 - szFileSize
0x4 - szFileZSize
0x4 - szUnknown01
0x4 - szOffset
0x1 - szComType(0 - Not Compressed, 1 - Zlib, 2 - Snappy)
0x1 - szArchiveNum
0x2 - szUnknown02


Hash Function: Initialize hash by 0xA8396u

http://rdf3x.googlecode.com/svn/trunk/i ... l/Hash.cpp
http://rdf3x.googlecode.com/svn/trunk/i ... l/Hash.hpp


Last edited by Ekey on Tue Aug 20, 2013 11:16 am, edited 1 time in total.

Top
 Profile  
 
 Post subject: Re: Researching of a .dat archive with a .mnf file.
PostPosted: Tue Aug 20, 2013 11:15 am 
Offline
veteran

Joined: Fri Aug 19, 2005 4:31 pm
Posts: 112
Has thanked: 14 times
Have thanks: 32 times
Exactly, but it might not be linked with the filenames. What if block 1 or 2 (or the extra data after block 3) contains that link and block 3 just contains sizes and offsets?


Top
 Profile  
 
 Post subject: Re: Researching of a .dat archive with a .mnf file.
PostPosted: Tue Aug 20, 2013 11:18 am 
Offline
M-M-M-Monster veteran
M-M-M-Monster veteran

Joined: Wed Mar 31, 2010 6:54 am
Posts: 1593
Has thanked: 59 times
Have thanks: 748 times
In 3 block - 0x4 - szUnknown01 - it's seems filename hash (see my previous post)


Top
 Profile  
 
 Post subject: Re: Researching of a .dat archive with a .mnf file.
PostPosted: Tue Aug 20, 2013 1:51 pm 
Offline
veteran

Joined: Fri Aug 19, 2005 4:31 pm
Posts: 112
Has thanked: 14 times
Have thanks: 32 times
Yes, that will help a lot once we have found the complete list of filenames, but we still need to find that, right?

By the way, these are the 15 unknown bytes after the MES2 header:
Code:
             unknown1  num_dat_archives       EOF_offset           dummy         unknown2
FILENAME         [02 00] [XX] 00 00 00 00 [XX XX XX XX] 00 03 [00 00 00] 04 [XX XX XX XX]
eso.mnf:         [02 00] [D0] 00 00 00 00 [7B FB 4F 00] 00 03 [00 00 00] 04 [00 08 0C 01]
esoaudioen.mnf:  [02 00] [04] 00 00 00 00 [2B 09 1E 00] 00 03 [00 00 00] 04 [00 04 06 01]
game.mnf:        [02 00] [01] 00 00 00 00 [3A A2 00 00] 00 03 [00 00 00] 04 [00 00 10 19]

unknown1: This always has to be 2 for some reason. The files don't open otherwise.
num_dat_archives: number of DAT files
EOF_offset (little endian): After reading these 4 bytes, add EOF_offset to current offset to reach the end of the third block. This is EOF except for eso.mnf which has extra data.
unknown2 (big endian): This is 4120*(2^n)+1, with n ranging from 0 to 2. It seems n is also the number of the DAT archive containing the ZOSFT, but maybe this is just a coincidence.


EDIT -> I'm in the process of decompiling eso.exe. It's not going that well as expected but I fixed a few things in the description above.


Top
 Profile  
 
 Post subject: Re: Researching of a .dat archive with a .mnf file.
PostPosted: Fri Oct 04, 2013 10:31 am 
Offline
veteran

Joined: Fri Aug 19, 2005 4:31 pm
Posts: 112
Has thanked: 14 times
Have thanks: 32 times
Bumping this topic.
About a month and a half has passed. I got stuck in the decompilation process.
Has anybody made any progress?

Here are my scripts so far. My code is a bit messy but then again, QuickBMS isn't that clean as a language.

TERAB1T, it would be great if you could share how you linked the ZOSFT filenames to the files. :)


Top
 Profile  
 
 Post subject: Re: Researching of a .dat archive with a .mnf file.
PostPosted: Sat Oct 12, 2013 2:34 am 
Offline
n00b

Joined: Mon May 28, 2012 12:40 am
Posts: 13
Has thanked: 1 time
Have thanks: 5 times
About game.mnf filenames:

There are three zlib-ed blocks in .mnf file. As you know, the last one contains filesizes, offsets etc (20 bytes for each file). And the second one contains indexes for some of filenames from zosft-file: 8 bytes for each file. Look at this screenshot:

Image

If 'Unknown1 = 0' and 'Unknown3 = 128 (decimal)' index (Unknown0) will be valid.
And 'Unknown1 = 1' and 'Unknown3 = 128' is filetable (zosft).

Use valid indexes and link them with files at the end of filetable (after 9 zlibed blocks).

Also I figured out, that if you remove all the "00 00 00 00" parts from the first block in .mnf, you'll get unique ID for each file in archives (4 bytes for each file).


Top
 Profile  
 
 Post subject: Re: Researching of a .dat archive with a .mnf file.
PostPosted: Sun Oct 13, 2013 12:47 pm 
Offline
veteran

Joined: Fri Aug 19, 2005 4:31 pm
Posts: 112
Has thanked: 14 times
Have thanks: 32 times
Interesting - there are only 1768 valid indices in the MNF, although there are 1772 records in the ZOSFT file. It's not so hard to combine those indices with the ZOSFT without checking their validity though: they return in ZOSFT block 2.3 (that is, the third zlib block of the second big block, so it's the 6th zlib block), along with offsets for the corresponding filenames.
The indices are also unique in game.mnf, while they aren't in eso.mnf... quite strange indeed. I think that when combined with another number (which isn't necessary for game.mnf), they might be unique.

I had also found out that there are unique IDs for the files in the first MNF block (check the ZOSFT script in my previous post, it used them), and these IDs appear again in a few of the ZOSFT blocks, but they seem to be in the wrong order. The indices in block 2 seem to be the only link. I called that index file_number in my research notes which I made available here: http://pastebin.com/rsgh1irr

I adapted my scripts so the filenames of game.zosft are correctly applied. :)


Top
 Profile  
 
 Post subject: Re: Researching of a .dat archive with a .mnf file.
PostPosted: Sun Nov 24, 2013 10:00 am 
Offline
beginner

Joined: Mon Jun 25, 2012 8:44 am
Posts: 20
Has thanked: 44 times
Have thanks: 0 time
Hey guys, anyone succeeded at extracting the sounds ? eso0000.dat through eso0207.dat should contain sfx. And esoaudioen0000.dat through esoaudioen0003.dat got to be the container for the voices.

I have attached a sample of the voice archive HERE. (1,09 gb)
And also the esoaudioen.mnf, just in case.

And the possible sfx HERE.

Anyone gets anything, let me know. Thanks!


Top
 Profile  
 
 Post subject: Re: Researching of a .dat archive with a .mnf file.
PostPosted: Wed Nov 27, 2013 8:16 pm 
Offline
veteran

Joined: Fri Aug 19, 2005 4:31 pm
Posts: 112
Has thanked: 14 times
Have thanks: 32 times
Extracting the audio is easy, you can do it with my QuickBMS scripts to extract everything using the MNF files. You'd better just extract all the archives in one go though, because the files the MNF file points to is divided over all of those DAT files.
Psst! If you know how to use Google, you can also find the soundtrack which I ripped and posted online some time ago...


Top
 Profile  
 
 Post subject: Re: Researching of a .dat archive with a .mnf file.
PostPosted: Thu Jan 09, 2014 4:33 pm 
Offline
beginner

Joined: Sat Sep 12, 2009 11:33 am
Posts: 30
Has thanked: 10 times
Have thanks: 5 times
I believe eso.mnf header has changed. Looking at this

Droolie wrote:
             unknown1  num_dat_archives       EOF_offset           dummy         unknown2
FILENAME         [02 00] [XX] 00 00 00 00 [XX XX XX XX] 00 03 [00 00 00] 04 [XX XX XX XX]
eso.mnf:         [02 00] [D0] 00 00 00 00 [7B FB 4F 00] 00 03 [00 00 00] 04 [00 08 0C 01]
esoaudioen.mnf:  [02 00] [04] 00 00 00 00 [2B 09 1E 00] 00 03 [00 00 00] 04 [00 04 06 01]
game.mnf:        [02 00] [01] 00 00 00 00 [3A A2 00 00] 00 03 [00 00 00] 04 [00 00 10 19]


I do not think bytes are in this order anymore. (there are extra bytes now and some are not all 00)


Top
 Profile  
 
 Post subject: Re: Researching of a .dat archive with a .mnf file.
PostPosted: Fri Jan 10, 2014 12:49 pm 
Offline
beginner

Joined: Sat Sep 12, 2009 11:33 am
Posts: 30
Has thanked: 10 times
Have thanks: 5 times
It seems that that unknown block of data at the end of eso.mnf you mentioned previously is now inserted after the first 22 bytes. I'll try and create a functioning bms script to compensate for this, but I'm a noob so ...


Top
 Profile  
 
 Post subject: Re: Researching of a .dat archive with a .mnf file.
PostPosted: Fri Jan 10, 2014 4:26 pm 
Offline
beginner

Joined: Sat Sep 12, 2009 11:33 am
Posts: 30
Has thanked: 10 times
Have thanks: 5 times
For the latest build things seem to have changed in eso.mnf. Block of dat that used to be after the third zlib block is now moved to 22nd byte. I modified Droolie's quickbms script to compensate for this change. I might have broken something else, but I did manage to extract a lot of DDS textures and gr2 models this way. WAVs seem to be broken (won't play in media players), but it might be something else.

There does not seem to be a ZOSFT file extracted with this script from eso*.dat files so I could not use the 2nd Droolie script to connect filenames with files. Any help is appreciated.


Attachments:


You do not have the required permissions to view the files attached to this post. Register to gain access.



Top
 Profile  
 
 Post subject: Re: Researching of a .dat archive with a .mnf file.
PostPosted: Fri Jan 10, 2014 7:52 pm 
Offline
ultra-n00b

Joined: Sun Mar 25, 2012 4:34 pm
Posts: 1
Has thanked: 0 time
Have thanks: 0 time
Thanks for all the great work so far. Does anyone have a hint what type of packer/obfuscator was used for the current stress test beta exo.exe ?


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 63 posts ]  Go to page Previous  1, 2, 3, 4, 5  Next

All times are UTC + 1 hour


Who is online

Users browsing this forum: wlscks7, z22901206 and 7 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group