[Request] Guild Wars 2 DAT

The Original Forum. Game archives, full of resources. How to open them? Get help here.
Post Reply
Nimba
beginner
Posts: 38
Joined: Wed Mar 07, 2012 8:32 pm
Been thanked: 14 times

[Request] Guild Wars 2 DAT

Post by Nimba » Wed Mar 07, 2012 8:37 pm

I wanted to enlist some help on getting the DAT file for the GW2 client figured out. It shares similarities to GW1 from what I see, but its different, and I couldn't quite figure it out.

I can provide the 16mb patcher that you download, it downloads the rest of the game. I can also provide cuts of the archive file (what file cutter do you recommend?).

Just tell me which is the preferred route and how to get it to you guys.

Ekey
M-M-M-Monster veteran
M-M-M-Monster veteran
Posts: 1767
Joined: Wed Mar 31, 2010 6:54 am
Has thanked: 90 times
Been thanked: 984 times

Re: [Request] Guild Wars 2 DAT

Post by Ekey » Wed Mar 07, 2012 9:19 pm

aluigi wrote:and where are the samples?
aluigi wrote:If files big use filecutter or other solutions:
http://www.xentax.com/downloads/tools/filecutter.zip
PS: Or just link for download client.

Nimba
beginner
Posts: 38
Joined: Wed Mar 07, 2012 8:32 pm
Been thanked: 14 times

Re: [Request] Guild Wars 2 DAT

Post by Nimba » Wed Mar 07, 2012 9:40 pm

* snip *

That is the link to the client patcher. 5 mb (rar file), 16mb extracted, it will download the rest of the client.

It downloads in place, so put it into a convenient folder.

The full size of the DAT after downloading the full client is 16gb
Last edited by Mr.Mouse on Thu May 17, 2012 4:12 pm, edited 2 times in total.
Reason: Rules violation

Nimba
beginner
Posts: 38
Joined: Wed Mar 07, 2012 8:32 pm
Been thanked: 14 times

Re: [Request] Guild Wars 2 DAT

Post by Nimba » Wed Mar 07, 2012 10:00 pm

I did do a bit of work already from the limited knowledge I know.

I also think there is a master file numbered 13525, it seems to have just a massive list in it. Not sure how its coded. URLs to their CDN are in the EXE if you run it through OllyDBG or monitor traffic with Wireshark. All files are downloaded compressed, but if you remove the compressed tag from the URLs, you will get the Uncompressed versions added to the .DAT file. If you get the cookie information, you can download them directly.

Other than that, I thought the list of files would be somewhere in memory, but I could not figure out where yet. So not sure how much this helps.

Ekey
M-M-M-Monster veteran
M-M-M-Monster veteran
Posts: 1767
Joined: Wed Mar 31, 2010 6:54 am
Has thanked: 90 times
Been thanked: 984 times

Re: [Request] Guild Wars 2 DAT

Post by Ekey » Thu Mar 08, 2012 1:25 pm

Cutted http://www.mediafire.com/?hsowrca6g562gzc

Code: Select all

Signsrch 0.1.7
by Luigi Auriemma
e-mail: aluigi@autistici.org
web:    aluigi.org
  optimized search function from Andrew http://www.team5150.com/~andrew/
  disassembler engine from Oleh Yuschuk

- open file "d:\GW2\Gw2Dev.exe"
- 22800384 bytes allocated
- load signatures
- open file d:\!Project\Game Tools\Aluigi\signsrch.sig
- 1774060 bytes allocated for the signatures
- 2278 signatures in the database
- WARNING:
  the file loaded in memory is very big so the scanning could take many time
- start signatures scanning:

  offset   num  description [bits.endian.size]
  --------------------------------------------
  0109fda0 31   Adler CRC32 (0x191b3141) [32.le.1024]
  010a0da0 32   Adler CRC32 (0x191b3141) [32.be.1024]
  010a01a0 33   Adler CRC32 (0x01c26a37) [32.le.1024]
  010a11a0 34   Adler CRC32 (0x01c26a37) [32.be.1024]
  010a05a0 35   Adler CRC32 (0xb8bc6765) [32.le.1024]
  010a15a0 36   Adler CRC32 (0xb8bc6765) [32.be.1024]
  00f66cf0 115  ADPCM index table (step variation) [32.le.64]
  00f66d30 117  ima_adpcm step table [16.le.178]
  01044940 135  libavutil ff_log2_tab [..256]
  0109e3e8 142  ACSS reverse sbox [..256]
  01022e48 145  SHA256 Hash constant words K (0x428a2f98) [32.le.256]
  008027e7 147  SHA256 Initial hash value H (0x6a09e667UL) [32.le.32&]
  010adfa0 167  Rijndael Te0 (0xc66363a5U) [32.le.1024]
  010ae3a0 169  Rijndael Te1 (0xa5c66363U) [32.le.1024]
  010ae7a0 171  Rijndael Te2 (0x63a5c663U) [32.le.1024]
  010aeba0 173  Rijndael Te3 (0x6363a5c6U) [32.le.1024]
  010aefa0 175  Rijndael Te4 (0x63636363U) [32.le.1024]
  010af3a0 176  Rijndael Td0 (0x51f4a750U) [32.le.1024]
  010af7a0 178  Rijndael Td1 (0x5051f4a7U) [32.le.1024]
  010afba0 180  Rijndael Td2 (0xa75051f4U) [32.le.1024]
  010affa0 182  Rijndael Td3 (0xf4a75051U) [32.le.1024]
  010b03a0 184  Rijndael Td4 (0x52525252U) [32.le.1024]
  010b07a0 185  Rijndael rcon [32.le.40]
  00801ee2 307  SHA1 / SHA0 / RIPEMD-160 initialization [32.le.20&]
  010a21c0 357  Zlib dist_code [..512]
  00f48ac0 358  Zlib length_code [..256]
  010a24c0 360  Zlib base_length [32.le.116]
  010a2538 362  Zlib base_dist [32.le.120]
  0118c4cc 373  possible mask reset used in arithmetic compression [..9]
  0109c650 384  Jpeg dct 14 bit aanscales [16.le.128]
  0109c610 388  Jpeg dct AA&N scale factor [double.le.64]
  0114e29a 550  Windows CryptAcquireContext [..21]
  0049d86b 568  classical random incrementer 0x343FD 0x269EC3 [32.le.8&]
  00e3d940 815  libavcodec ff_zigzag_direct [..64]
  00f632b0 851  libavcodec ASF asf_header [..16]
  00f67d48 855  Generic bitmask table [32.le.128]
  00f63cb0 973  asf_content_desc_guid [..16]
  011974b0 1014 mp3lib huffman tab1 [16.le.14]
  01196a6c 1016 mp3lib huffman tab2 [16.le.34]
  011974c0 1018 mp3lib huffman tab3 [16.le.34]
  011974e4 1020 mp3lib huffman tab5 [16.le.62]
  01197524 1022 mp3lib huffman tab6 [16.le.62]
  01197568 1024 mp3lib huffman tab7 [16.le.142]
  011975f8 1026 mp3lib huffman tab8 [16.le.142]
  01197688 1028 mp3lib huffman tab9 [16.le.142]
  01197718 1030 mp3lib huffman tab10 [16.le.254]
  01197818 1032 mp3lib huffman tab11 [16.le.254]
  01197918 1034 mp3lib huffman tab12 [16.le.254]
  01197a18 1036 mp3lib huffman tab13 [16.le.1022]
  01197e18 1038 mp3lib huffman tab15 [16.le.1022]
  01198218 1040 mp3lib huffman tab16 [16.le.1022]
  01198618 1042 mp3lib huffman tab24 [16.le.1022]
  01198a18 1044 mp3lib huffman tab_c0 [16.le.62]
  01198a58 1046 mp3lib huffman tab_c1 [16.le.62]
  01196a90 1048 mp3lib bandInfo [..2592]
  01194970 1049 mp3lib intwinbase [32.le.1028]
  00f67f68 1176 Vorbis FLOOR1_fromdB_LOOKUP [float.le.1024]
  0109e3e8 1198 FFT and FHT routines rv_tbl [..128]
  00f4b788 1228 rfc3548 Base 64 Encoding with URL and Filename Safe Alphabet [..
62]
  00f61648 1237 B64EncodeTable [..64]
  010a1b78 1525 zinflate_lengthExtraBits [32.le.116]
  010a1c00 1529 zinflate_distanceExtraBits [32.le.120]
  0114cc02 1767 anti-debug: IsDebuggerPresent [..17]
  00f67d4c 1810 bitmask [32.le.128]
  015be7b4 2253 PADDINGXXPADDING [..16]
  00f48bc0 2254 unlzx table_one [..32]
  00f67d48 2259 unlzx table_three [32.le.64]
  00de0bd0 2269 DMC compression [32.le.16&]
  0104d163 2271 compression algorithm seen in the game DreamKiller [32.le.12&]
  01046217 2272 compression algorithm seen in the game DreamKiller [32.be.12&]

- 70 signatures found in the file

Nimba
beginner
Posts: 38
Joined: Wed Mar 07, 2012 8:32 pm
Been thanked: 14 times

Re: [Request] Guild Wars 2 DAT

Post by Nimba » Fri Mar 09, 2012 5:03 am

So this is what I have found so far that could help:

/latest/102 : has the latest build # (ex. returns: 13525 2312313 12313123) the first number is the build number.
/manifest/program/102/{build#} : manifest file for build number (ex. 13525, 13603)
/program/102/0/0/{fileNumber}/ : the actual data file uncompressed (ex. 844474 - MP3 File of a Roar)
/program/102/0/0/{fileNumber}/compressed : the actual data file compressed (have not investigated the compression used)

The Manifest File:

It is a list of UInt32 in Little Endian order. once converted to decimals, they are the fileNumber used in the above links. They do not have filenames, the number is the filename.
The Cookie information, and the CDN address is in the GW2Dev.exe file, should not be hard to find for those that are working on it.

Also, randomly (well seems random to me) there are numbers in the Manifest files that are not Files. They are the size in bytes of the previous file downloaded.

Example:
1814180 - 345208 bytes
2519389 - 345036 bytes
345036
2054826 - 72964 bytes
2948595 - 104384 bytes
104384

There are a lot of files unknown to me. AMATGMRT, MAPCPARM, PIMGPG, AMSPAMSP, MODLMODL, CINPC, and hvkChavK. A lot of files seem to have messed up or modified headers. like a good portion of the DXT files. They have a ATEXDXT# in the header, instead of some of the ones that work that start with DDS|. I have no clue how to repair the header, if its even possible. But I did some reading, it sounds similar to the GW1 ATEXDXT files. Messed up header, seems the size of the image is there somewhere after the header. Missing Texels? Thats all I read.

Nimba
beginner
Posts: 38
Joined: Wed Mar 07, 2012 8:32 pm
Been thanked: 14 times

Re: [Request] Guild Wars 2 DAT

Post by Nimba » Wed Mar 14, 2012 12:11 am

Anyone have any success or progress on this? Just curious.

User avatar
chrrox
Moderator
Posts: 2602
Joined: Sun May 18, 2008 3:01 pm
Has thanked: 57 times
Been thanked: 1362 times

Re: [Request] Guild Wars 2 DAT

Post by chrrox » Wed Mar 14, 2012 1:14 am

i got a script to look at the uncompressed files but it downloaded 50 gigs of files not sure how that's possible.

Nimba
beginner
Posts: 38
Joined: Wed Mar 07, 2012 8:32 pm
Been thanked: 14 times

Re: [Request] Guild Wars 2 DAT

Post by Nimba » Wed Mar 14, 2012 1:24 am

chrrox wrote:i got a script to look at the uncompressed files but it downloaded 50 gigs of files not sure how that's possible.
Well, I dunno what compression they use, or if they have secondary manifests somewhere, but that is correct. Uncompressed size of the download is 50 gigs of files. But compressed it is 16 gigs inside the DAT file. It compresses each file individually and then inserts it into the dat.

If you modify the GW2Dev.exe file and remove the /compressed from the URLs that are coded into the EXE, it will download 50 gigs directly into the DAT file. At least it did the last time I checked.

You can try it yourself and see. Backup your 16 gig file, and download the 50 gig version. you can then use it to compare for finding how files are stored in the DAT file? and then you can compared the uncompressed to the compressed to see how ti was compressed.

Not sure what to tell you.

User avatar
chrrox
Moderator
Posts: 2602
Joined: Sun May 18, 2008 3:01 pm
Has thanked: 57 times
Been thanked: 1362 times

Re: [Request] Guild Wars 2 DAT

Post by chrrox » Wed Mar 14, 2012 1:45 am

now that would be very usefull especially if it downloads them in order.
ill give that a try.

Nimba
beginner
Posts: 38
Joined: Wed Mar 07, 2012 8:32 pm
Been thanked: 14 times

Re: [Request] Guild Wars 2 DAT

Post by Nimba » Wed Mar 14, 2012 1:57 am

chrrox wrote:now that would be very usefull especially if it downloads them in order.
ill give that a try.
You can also try with small samples. Let it download 20-30 files uncompressed, then 20-30 comrpessed into seperate dat files. So you dont have to wait.

You can then also check if they are in order.

Also you can download the individual files with my script, or a modified version of my script, and find the data in the archives. I dont have the tools to do it, but maybe some how search for the contents of Uncompressed File 1 in Uncompressed DAT, and then search Compressed File 1 in Uncompressed DAT file. and it could help you figure out how the DAT is split up and so on.

I think from there you know more than I do.

I also believe it downloads them in the order of the manifest.

Nimba
beginner
Posts: 38
Joined: Wed Mar 07, 2012 8:32 pm
Been thanked: 14 times

Re: [Request] Guild Wars 2 DAT

Post by Nimba » Sun Mar 18, 2012 8:29 pm

Any luck?

User avatar
chrrox
Moderator
Posts: 2602
Joined: Sun May 18, 2008 3:01 pm
Has thanked: 57 times
Been thanked: 1362 times

Re: [Request] Guild Wars 2 DAT

Post by chrrox » Sun Mar 18, 2012 9:55 pm

i never got the uncompressed data as one big file

Nimba
beginner
Posts: 38
Joined: Wed Mar 07, 2012 8:32 pm
Been thanked: 14 times

Re: [Request] Guild Wars 2 DAT

Post by Nimba » Mon Mar 19, 2012 1:48 am

chrrox wrote:i never got the uncompressed data as one big file
Even after modifying the patcher?

At address 00F5F454 of the latest exe (Build 13703), there is a line that says /compressed (2F 63 6F 6D 70 72 65 73 73 65 64), if you 0 that out, it should download everything uncompressed into the data file. I just did it yesterday to test. I have a 34 gig DAT file atm. Still downloading.

[Edit] Just did it. First ~1000 files. Compressed file is 135MB, uncompressed file is 239MB. I am uploading them both atm to dropbox and will link them to you in a bit.

Nimba
beginner
Posts: 38
Joined: Wed Mar 07, 2012 8:32 pm
Been thanked: 14 times

Re: [Request] Guild Wars 2 DAT

Post by Nimba » Mon Mar 19, 2012 2:58 am

Compressed DAT Part (132MB) :Gw2DevStream-Compressed.dat ( just the first ~1000 files downloaded)
Uncompressed DAT Part (233MB): Gw2DevStream-Uncompressed.dat ( hopefully the same ~1000 files )

The way i cut it off, i started the patcher with no Data file (backed up my old one) and launched it, when it reached 157000 remaining files, I shut off the patcher. and I believe it started at 158000ish. Then I 00'd out the /compressed in the EXE, and did the same with a new data file.

[Edit] I took down the links, so they aren't linked to my dropbox account, and I believe anyone that would need them, has them.

Post Reply