XeNTaX Forum Index
Forum MultiEx Commander Tools Tools Home
It is currently Wed Oct 18, 2017 10:22 am

All times are UTC + 1 hour


Forum rules


Please click here to view the forum rules



Post new topic Reply to topic  [ 63 posts ]  Go to page Previous  1, 2, 3, 4, 5  Next
Author Message
 Post subject: Re: Researching of a .dat archive with a .mnf file.
PostPosted: Sat Jan 11, 2014 8:03 pm 
Offline
beginner

Joined: Sat Sep 12, 2009 11:33 am
Posts: 30
Has thanked: 10 times
Have thanks: 5 times














You can make the ads go away by registering

For whatever reason the quickbms script does not extract the zosft file from esodata files. I managed to find the file in eso0002.dat but it contains only a small portion of filenames (something like 30k as compared to over 250k present). It also seems that the quickbms script does not extract some additional files for whatever reason. I might be wrong though.


Top
 Profile  
 
 Post subject: Re: Researching of a .dat archive with a .mnf file.
PostPosted: Tue Jan 14, 2014 4:58 pm 
Offline
beginner

Joined: Sat Sep 12, 2009 11:33 am
Posts: 30
Has thanked: 10 times
Have thanks: 5 times
That bms script I posted several days ago will not properly unpack everything. Turns out it breaks when some of the empty archives are being called and eso0000.dat has files that are neither zlib nor snappy ( or something in the original bms script is not quite correct). Currently, I am skipping files that cause breaks in the script manually, but any help from the more experienced members would be welcome.


Top
 Profile  
 
 Post subject: Re: Researching of a .dat archive with a .mnf file.
PostPosted: Wed Jan 15, 2014 1:39 pm 
Offline
ultra-n00b

Joined: Fri Dec 31, 2010 12:54 am
Posts: 4
Has thanked: 0 time
Have thanks: 0 time
Heya,

Never been active on this forum a lot, always lurking and messing around by myself. Let my extraction program run for a good half an hour yesterday , it would go through all archives, decompress the information, and check the header against a local file. I had to skip eso0000 and everything beyond eso0106, because there's stuff in there which I suspect is double compressed (so first Zlib then Snappy or other way around, because those files have multiple entries in the second block of the MNF with different "flags").

There aren't a lot of special files in there though: .xml, .lua, .dds, .gr2, some soundbanks, some audio files, a few custom files and thats about it.

Besides a single ZOSFT in all the scanned eso packages I also encountered a ANFT, which I suspect is some sort of internal filetable. Haven't been able to find another ZOSFT, but it could be its hidden away in side the packages I skipped.

However, as someone in this topic said before: It's not unlikely some files are only used interally.

_________________
Disclaimer: I reverse formats for sports only, I have nothing to gain of it but information on how they work and a feeling of accomplishment... much like finally defeating a pesky boss in Dark Souls :D


Top
 Profile  
 
 Post subject: Re: Researching of a .dat archive with a .mnf file.
PostPosted: Wed Jan 15, 2014 7:08 pm 
Offline
ultra-n00b

Joined: Fri Dec 31, 2010 12:54 am
Posts: 4
Has thanked: 0 time
Have thanks: 0 time
Attached is a rar with logs for all archives.

It's safe to say that Block 3 in the MNFs contains all entries into the DAT files as all bytes are consumed when going through them (except 8 bytes in archive 4 for some reason). The MNF for ESO also contains references to archive 0 which don't exist, so either its residual or it's used for whatever more is in the MNF (eg. the unknown block).

All archives beyond 203 are simply empty, they only have the first 14 bytes of the DAT header. However, the MNF contains over 50k entries into these files, so I guess the content is supposed to be there, but it's stripped/disabled for the beta.

Other than that, as said in my previous post, not a lot of interesting things going on in the files.

With these logs, you/I can check which files are in the ZOSFT to give them a name and which are most likely for internal use only.

EDIT: Logs.rar now also contains list of all filenames found in the ZOSFT, sorted by filename and extension. As you can see, a lot of the files you encounter in the other logs are not mentioned in the ZOSFT.


Attachments:


You do not have the required permissions to view the files attached to this post. Register to gain access.


_________________
Disclaimer: I reverse formats for sports only, I have nothing to gain of it but information on how they work and a feeling of accomplishment... much like finally defeating a pesky boss in Dark Souls :D


Top
 Profile  
 
 Post subject: Re: Researching of a .dat archive with a .mnf file.
PostPosted: Thu Jan 16, 2014 1:38 pm 
Offline
beginner

Joined: Sat Sep 12, 2009 11:33 am
Posts: 30
Has thanked: 10 times
Have thanks: 5 times
I came to the same conclusions working with the bms scripts. There is one more thing I'd like to add though - file id is not unique. There are some files that have the same ID but are from diferent archives. So, for example, file with id 9 can be found in eso0000.dat and eso000x.dat and so on. I did not manage to find the correlation between archive_id, file_id and zosft so that I can determine which file from which archive is connected to which filename.


Top
 Profile  
 
 Post subject: Re: Researching of a .dat archive with a .mnf file.
PostPosted: Fri Jan 31, 2014 1:12 am 
Offline
advanced

Joined: Tue Feb 28, 2012 11:21 pm
Posts: 70
Has thanked: 11 times
Have thanks: 3 times
While follow this, it looks like ESO use a strong method to save their files.


Top
 Profile  
 
 Post subject: Re: Researching of a .dat archive with a .mnf file.
PostPosted: Thu Feb 06, 2014 2:20 pm 
Offline
veteran

Joined: Fri Aug 19, 2005 4:31 pm
Posts: 112
Has thanked: 14 times
Have thanks: 32 times
After the latest update, it seems that as LennardF1989 said, eso0000.dat is compressed differently, or the MNF records are just wrong for that file. I haven't got the time to analyze it properly, so is anyone interested?

I'm currently using this adapted script to extract the audio and the ZOSFT only (since I'm only interested in those). You can easily adapt it (by using my older scripts) to extract all the files. It only extracts files in archives with 0 < archive number < 204. You can remove those restrictions to be able to extract files from MNFs other than eso.mnf. ;)


Top
 Profile  
 
 Post subject: Re: Researching of a .dat archive with a .mnf file.
PostPosted: Fri Feb 07, 2014 8:44 pm 
Offline
ultra-n00b

Joined: Fri Dec 31, 2010 12:54 am
Posts: 4
Has thanked: 0 time
Have thanks: 0 time
I purposely waited for the next ESO weekend, because it seems it's not a mature format yet :)

_________________
Disclaimer: I reverse formats for sports only, I have nothing to gain of it but information on how they work and a feeling of accomplishment... much like finally defeating a pesky boss in Dark Souls :D


Top
 Profile  
 
 Post subject: Re: Researching of a .dat archive with a .mnf file.
PostPosted: Sat Feb 15, 2014 2:38 pm 
Offline
ultra-n00b

Joined: Sat Feb 15, 2014 1:38 pm
Posts: 1
Location: France
Has thanked: 1 time
Have thanks: 0 time
I've been lurking this post for a while, and I must say great work!
I'm only interested in the .dds files and have been succesfully getting plenty of those in the eso.mnf/eso00xxx.dat files.

But I don't seem to get the same result for the game.mnf one.
The topic started with the game.mnf file and shifted to the eso.mnf file.
Does anyone have a bms for the game.mnf one? I'm still pretty much a n00b when i comes to figuring it out.
I don't care about the file names, as long as the dds files are exported.
TERAB1T uploaded an exported one, but it seems to be missing a lot of files (perhaps due to not getting the right filenames)
Anyone that can help?


Top
 Profile  
 
 Post subject: Re: Researching of a .dat archive with a .mnf file.
PostPosted: Fri Feb 28, 2014 9:18 pm 
Offline
ultra-n00b

Joined: Sun May 20, 2012 3:56 pm
Posts: 4
Has thanked: 0 time
Have thanks: 0 time
None of the QuickBMS scripts seem to work, they end up with xmalloc errors. I think incorrect values are being used to extract files. Anyone have working scripts still?


Top
 Profile  
 
 Post subject: Re: Researching of a .dat archive with a .mnf file.
PostPosted: Tue Mar 11, 2014 9:06 pm 
Offline
beginner

Joined: Sun Jul 14, 2013 10:28 pm
Posts: 37
Has thanked: 6 times
Have thanks: 14 times
Extracting main game archives is the first step but what about rev eng data/files inside?
Was any of you able to find structure of any, non simple, files inside?
There are maps (gfx & names), localized text (en/fr/de) containing quests data (titles, descriptions, objectives), lore books texts, npc names, items, names, etc. and even some ids, textures, sounds, ui scripts, etc.
But I'm still missing "connections" between those data - some kind of internal database, i.e. I know quest ids, names and descriptions but I'm missing quest level and category, or connection between quest and its objectives.
Or I'm not able to find connection between map gfx and map id - I know that for most of the zones its simple (just one map and gfx), but multi level zones (i.e the wailing prison) are the problem.

Have any of you found such kind of data? :-)


Top
 Profile  
 
 Post subject: Re: Researching of a .dat archive with a .mnf file.
PostPosted: Sat Mar 15, 2014 6:20 pm 
Offline
beginner

Joined: Sat Sep 12, 2009 11:33 am
Posts: 30
Has thanked: 10 times
Have thanks: 5 times
None of that has been publicly reverse engineered. No one has yet delved into decrypting eso0000.dat and eso0001.dat, which contain a lot of files not in zosft files and are compressed with an unknown method (not zlib, not snappy). Looking at some websites out there I can see that probably noone reverese engineered the client and they are all sniffing packets or using in-game addons for data collection. I guess there is not enough interest by the pros to go deep into how zenimax encrypted their files.


Top
 Profile  
 
 Post subject: Re: Researching of a .dat archive with a .mnf file.
PostPosted: Mon Mar 17, 2014 11:47 am 
Offline
beginner

Joined: Sat Sep 12, 2009 11:33 am
Posts: 30
Has thanked: 10 times
Have thanks: 5 times
I was snooping around eso0000 and offzip-ed it. Bunch of files with header
Code:
fa fa eb eb
or
Code:
FB FB EC EC
One of these have multiple zlibed blocks, while other seems to be and index of sorts. When I offzipped the one with zlib blocks I got thousands of new files. Some of them contain some strings and I believe this has something to do with map names and their IDs (possible map tiles?) and possible quest/NPC names (possible map locations?). I also found something strange which I am not able to decipher on my own yet. There are files that apparently have some sort of encrypted or compressed strings. From my crude analysis the files are composed of a header followed by the number of strings contained, followed by 8 byte (Id?), followed by 00, followed by str_length, followed by string, followed by 00.

Attached is one of those files. Does anyone recognize what this is compressed/encrypted with?


Attachments:


You do not have the required permissions to view the files attached to this post. Register to gain access.



Top
 Profile  
 
 Post subject: Re: Researching of a .dat archive with a .mnf file.
PostPosted: Fri Mar 21, 2014 2:12 am 
Offline
beginner

Joined: Fri Feb 17, 2012 4:56 pm
Posts: 20
Has thanked: 0 time
Have thanks: 0 time
Shame, would be nice to see these models out. :)


Top
 Profile  
 
 Post subject: Re: Researching of a .dat archive with a .mnf file.
PostPosted: Sat Apr 19, 2014 7:57 pm 
Offline
ultra-n00b

Joined: Fri Apr 18, 2014 8:09 pm
Posts: 1
Has thanked: 0 time
Have thanks: 0 time
The Strings are encoded with somethink like Base64.

Here the first three links, decoded with Base64 and converted to hex.
Code:
0413787f1451300d0a2f59
047809043108273978044b496969391d64103d0f297b3f5d7a2f3e06480e480120035f01021820082a601055121a254405493040362f3f185e7662682224743d130c6f481b6c48010201064808475d234f
045d78636018050234674470471f471e5919076448123f241a6044474038115903043b443d07112e3c641f5c6040401e66611805581f3a386b


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 63 posts ]  Go to page Previous  1, 2, 3, 4, 5  Next

All times are UTC + 1 hour


Who is online

Users browsing this forum: Google [Bot] and 7 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group