XeNTaX Forum Index
Forum MultiEx Commander Tools Tools Home
It is currently Sun Oct 26, 2014 10:26 am

All times are UTC + 1 hour


Forum rules


Please click here to view the forum rules




Post new topic Reply to topic  [ 516 posts ]  Go to page 1, 2, 3, 4, 5 ... 35  Next
Author Message
 Post subject: .wz archive
PostPosted: Thu Mar 02, 2006 9:20 am 
Offline
beginner

Joined: Sun Feb 26, 2006 3:16 am
Posts: 36
Has thanked: 0 time
Have thanks: 0 time
Hi,

I am trying to figure out how to extract maple story's wz file archive. I used Nova Extractor and found the positions of mp3s and I am using that as a stepping stone to figure out the rest of the archive. I've manage to save a block between 265210743 to 266877208, 266877209 is the start of another mp3, and it successfully played!

The file size indicates that 1666351 bytes, but from 265210743 to 266877208 the file size is actually 1666466 bytes, so right now I am confused : P
I found the file size at 265210652 in 4 bytes? I am not sure. ( Sorry, I can't stand to work in Hex offsets, thus my noob approach : P ) I actually have the file's name, but I guess I first need to find where the file name actually begins first.

Can someone please provide some insight, comments, what should I look for? The game can be downloaded from http://download.mapleglobal.com/downloa ... SSetup.exe It's around 314mb.

How does Nova Extractor recognize that there are mp3 files at that position, but doesn't detect anything else? Since I can extract the block does that mean it's not compressed?


Last edited by Meds on Sun Apr 16, 2006 8:27 pm, edited 1 time in total.

Top
 Profile  
 
 Post subject:
PostPosted: Thu Mar 02, 2006 9:43 am 
Offline
Site Admin
User avatar

Joined: Wed Jan 15, 2003 6:45 pm
Posts: 3742
Location: Dungeons of Doom
Has thanked: 309 times
Have thanks: 429 times
No, I just think the extractor looks for signatures that identify a mp3 file.

anyway, will take a look. :D

_________________
Game Request Rules


Top
 Profile  
 
 Post subject:
PostPosted: Thu Mar 02, 2006 7:04 pm 
Offline
beginner

Joined: Sun Feb 26, 2006 3:16 am
Posts: 36
Has thanked: 0 time
Have thanks: 0 time
You will? Awesome, I am eager to hear what your insights are for this.


Top
 Profile  
 
 Post subject:
PostPosted: Fri Mar 03, 2006 2:23 am 
Offline
ultra-n00b

Joined: Fri Mar 03, 2006 1:57 am
Posts: 5
Has thanked: 0 time
Have thanks: 0 time
I've been lookin' through Data.wz a bit...and Nova Extractor finds the mp3s cause they're all uncompressed in the archive.

I did manage to find alot of image data...there are alot of zlib compressed blocks in the archive...but they all have invalid block lengths. I figure they did this on purpose, because each block is preceded by four bytes which are the extact size of the compressed zlib block, and then a 0 byte.

For instance, at offset 225940914 in Data.wz there is a zlib compressed block...the 5 bytes before it contain the size, 5928 bytes and that 0 byte. When I extracted that block (and stopped before zlib reported an invalid stored block length error) I found out it contained 16 bit image data. I'm just not exactly sure what format it is in. By messing around with it a bit I managed to fit it into a 258 by 172 pixel bitmap of a golem from the game, but the colors are all wrong. I also found the values 258 and 172 just before the zlib block, but I can't seem to find any sort of regular pattern between all the zlib blocks. When I extract some of the images, they end up all distorted...so I'm wondering if they may have split some of the images into many zlib blocks.

Anyway, hope some of this info is useful...and I'd be interested if anyone has any ideas on what format the image data might be in...I know it's 16 bit...at least for some of the images...but I don't know if it's truecolor or palette or what...if it is palette, I'm not sure where the palette is :p


Top
 Profile  
 
 Post subject:
PostPosted: Fri Mar 03, 2006 7:30 am 
Offline
Site Admin
User avatar

Joined: Wed Jan 15, 2003 6:45 pm
Posts: 3742
Location: Dungeons of Doom
Has thanked: 309 times
Have thanks: 429 times
Good work! Did you also find some kind of table in there pointing to the location of the individual files in the archive?

_________________
Game Request Rules


Top
 Profile  
 
 Post subject:
PostPosted: Fri Mar 03, 2006 10:02 am 
Offline
beginner

Joined: Sun Feb 26, 2006 3:16 am
Posts: 36
Has thanked: 0 time
Have thanks: 0 time
Valvwen, that's a great find! I tried to reproduce your findings but, I am sorry, I have a novice question:

How did you uncompress your extraction with the zlib library? Are there unpackers out there? Because, I search all over and the only solution would be to create a custom solution using the library.

I am curious to what your extraction looked like, if you don't mind, can you please post your results? Inquiring minds want to know : P


Top
 Profile  
 
 Post subject:
PostPosted: Fri Mar 03, 2006 3:15 pm 
Offline
ultra-n00b

Joined: Fri Mar 03, 2006 1:57 am
Posts: 5
Has thanked: 0 time
Have thanks: 0 time
I haven't managed to find a file table. I searched for the offsets, but haven't found any for the files themselves, and I've had trouble finding the beginning of each file header, they seem to be variable length, which makes me think that they contain an encrypted filename. For instance, infront of every mp3 (the entire second half of the archive after offset 265979867 seems to be mp3s) is a variable length set of bytes(could be the filename?), then the bytes 0x0080 then the file size. After that it's a variable length of bytes until you get to an mp3 header (0xFFF29040 or any number of valid mp3 headers). For about the first 50 mp3 files after that, the length of bytes after the file size is 87, but then it changes.

As for finding and uncompressing the zlib blocks, I used python and hex workshop. I got the idea for using python from an article I read at OpenRCE about extracting everquest 2 archives...but I had to modify it to go faster and find invalid archives. The original script iterated through all the bytes in the archive and tried to uncompress the whole thing...which really slow when the archive is 300+ meg.

I found all the zlib blocks with this script:
Code:
import mmap
import os
import sys
import zlib

f = open("Data.wz", "rb")

start = 0
f.seek(0, 2)
size = f.tell()
f.seek(0)
obj = zlib.decompressobj()
buf = ""

while start < size:
   offset = start
   obj = zlib.decompressobj()
   buf = ""
   try:
      while offset < size:
         d = obj.decompress(f.read(1))
         if len(obj.unused_data) > 0:
            break
         buf += d
         offset += 1
   except zlib.error, details:
      if str(details) == "Error -3 while decompressing: invalid stored block lengths" and len(buf) > 0:
         print "offset:", start
         print "compressed:", offset - start - 1, "uncompressed:", len(buf)
      start += 1
      f.seek(start)
      continue
   print "offset:", start
   print "compressed:", offset - start - 1, "uncompressed:", len(buf)
   start += 1
   f.seek(start)

f.close()


After that I go to the offset, look 5 bytes before to get the file size...then go back to the offset and cut out the data to a new file with hex workshop.
Then I use this script to uncompress the zlib data.
Code:
import mmap
import os
import sys
import zlib

if len(sys.argv) < 2:
   print "usage: extract.py <file>"
   sys.exit(1)

offset = 0

fd = os.open(sys.argv[1], os.O_RDONLY | os.O_BINARY)
m = mmap.mmap(fd, 0, None, mmap.ACCESS_READ)

size = len(m)
obj = zlib.decompressobj()
buf = ""

try:
   while offset < size:
      d = obj.decompress(m[offset])
      if len(obj.unused_data) > 0:
         break
      buf += d
      offset += 1
except zlib.error, details:
   print "offset:", offset, "length:", offset - 1
   print details
   m.close()
   os.close(fd)
   sys.exit(1)

print "compressed:", offset - start - 1, "uncompressed:", len(buf)
f = open(sys.argv[1] + ".dump", "wb")
f.write(buf)
f.close()

m.close()
os.close(fd)


As for actually viewing the image data...I wrote a quick c# program that just loads the data into a 16 bit bitmap (I tried 8, 16, 24, and 32 bit...16 is the only format that gives me the right image dimensions), and lets me modify the width of the image until it looks right :p Since I haven't been able to find width and height for all the images (partially I think cause most of the images are split up into smaller chunks).

I'll try to add an attachment of what I got for the golem data...you'll clearly see the golem, but the colors are all wrong cause can't figure out the exact image format.


Attachments:


You do not have the required permissions to view the files attached to this post. Register to gain access.



Top
 Profile  
 
 Post subject:
PostPosted: Sat Mar 04, 2006 7:34 am 
Offline
beginner

Joined: Sun Feb 26, 2006 3:16 am
Posts: 36
Has thanked: 0 time
Have thanks: 0 time
Thanks a lot for sharing that python scripts they were a big help--I've outputted all the offsets into a file..a pretty big one at that. I spent a day trying to figure out how exactly I should go about viewing the files extracted, I've looked into opening it in Java's imageio, python's image module, they didn't seem to like the fact it didn't have any recognition of a valid image file. As a last ditched effort, I tried to just create an empty jpeg/png/bitmap/gif and merged the uncompressed data with it, in various positions--no luck.

The uncompressed blocks are pretty wierd, most of the extracted bytes are zero valued and fills the gap between non zeroed portions.

As for the filenames, I have a few known mp3 file names.
For version 0.18: (going off of nova extractor's offsets)
offset file size name
265210743 : 1666351 : SleepyWood.mp3
266877209 : 1376392 : FloralLife.mp3
268253714 : 728294 : GoPicnic.mp3
268982122 : 1907461 : Nightmare.mp3


Top
 Profile  
 
 Post subject:
PostPosted: Sat Mar 04, 2006 3:07 pm 
Offline
ultra-n00b

Joined: Fri Mar 03, 2006 1:57 am
Posts: 5
Has thanked: 0 time
Have thanks: 0 time
I wrote some quick python code to put the uncompressed data into a 16bit bitmap.

Code:
import struct
import sys

if len(sys.argv) < 3:
   print "usage: tobmp.py <file> <width>"
   sys.exit(1)

width = int(sys.argv[2])

f1 = open(sys.argv[1], "rb")
f1.seek(0, 2)
size = f1.tell()
f1.seek(0)

### write BITMAPFILEHEADER ###
f2 = open(sys.argv[1] + ".bmp", "wb")
f2.write("BM")

bfSize = size + 52
f2.write(struct.pack("L", bfSize))

# bfReserved1 and 2
f2.write("\x00\x00\x00\x00")

bfOffBits = 52
f2.write(struct.pack("L", bfOffBits))

### write BITMAPINFOHEADER ###
biSize = 40
f2.write(struct.pack("L", biSize))

biWidth = width
f2.write(struct.pack("L", biWidth))

biHeight = (size / 2 / width) + 1
f2.write(struct.pack("L", biHeight))

biPlanes = 1
f2.write(struct.pack("H", biPlanes))

biBitCount = 16
f2.write(struct.pack("H", biBitCount))

# biCompression
f2.write("\x00\x00\x00\x00")

# biSizeImage
f2.write("\x00\x00\x00\x00")

# biXPelsPerMeter
f2.write("\x00\x00\x00\x00")

# biYPelsPerMeter
f2.write("\x00\x00\x00\x00")

# biClrUsed
f2.write("\x00\x00\x00\x00")

# biClrImportant
f2.write("\x00\x00\x00\x00")

### write the bitmap data ###
data = f1.read()

# since bitmaps are upside down, gotta flip it over
i = len(data) - 1
while i >= 0:
   f2.write(data[i])
   i -= 1

# pad the end
padsize = (biWidth * biHeight * 2) - size
i = 0
while i < padsize:
   f2.write("\x00")
   i += 1

f1.close()
f2.close()


Hope it helps!


Top
 Profile  
 
 Post subject:
PostPosted: Mon Mar 06, 2006 3:21 am 
Offline
beginner

Joined: Sun Feb 26, 2006 3:16 am
Posts: 36
Has thanked: 0 time
Have thanks: 0 time
Thanks for creating that python script, I don't think it converts the uncompressed files to anything recognizable, but it does make'em into bmp.

I am still a bit perplexed at what exactly is the format these headerless uncompressed files are. I was wondering if you, Valvwen, can verify that this attached file is actually an image or something totally off the beaten path.
Thanks.


Top
 Profile  
 
 Post subject:
PostPosted: Mon Mar 06, 2006 2:01 pm 
Offline
ultra-n00b

Joined: Fri Mar 03, 2006 1:57 am
Posts: 5
Has thanked: 0 time
Have thanks: 0 time
Yeah, that particular offset doesn't seem to make any image, or if it is one, it may be distorted because it's part of a larger image. They're probably not all image blocks. I'd post some offsets of stuff I've found, but you're working on a more recent version of Data.wz, so the offsets may be off. I've managed to find lvl up text, a robot sort of thing, that golem, a few more golems that were all distorted, and some magic effects like power strike and stuff. Bear in mind that most of the blocks that are images look completely unrecognizable until you find the correct width. It's alot of trial and error until we figure out how to find the image width in the file header before we actually try random values until they work.


Top
 Profile  
 
 Post subject:
PostPosted: Mon Mar 06, 2006 8:36 pm 
Offline
beginner

Joined: Sun Feb 26, 2006 3:16 am
Posts: 36
Has thanked: 0 time
Have thanks: 0 time
Okay : P I can stop going crazy in trying to find an image in that offset then. Well, I am extracting from version 0.18 but I also have 0.19.

You know what...there have been others that have successfully extracted images/text out of the archive, but of course trying to get them to even give me a hint as to how it was done resulted in some serious, elitist flaming.

Notable examples
sauna.gibbed.us, this site's naming scheme reflects searchable values in memory when the game is played, however isn't searchable in hex.

And someone was able to get the animation into gifs
Edit: This site just used the extracted images from saua.gibbed.us and merged all the sequence.
dtrmp4.cursedsanctuary.com/ms/random/AnimatedGIFs/


Last edited by Meds on Mon Mar 06, 2006 10:25 pm, edited 1 time in total.

Top
 Profile  
 
 Post subject:
PostPosted: Mon Mar 06, 2006 9:01 pm 
Offline
beginner

Joined: Sun Feb 26, 2006 3:16 am
Posts: 36
Has thanked: 0 time
Have thanks: 0 time
[...]


Last edited by Meds on Tue Mar 07, 2006 11:50 am, edited 4 times in total.

Top
 Profile  
 
 Post subject:
PostPosted: Mon Mar 06, 2006 9:52 pm 
Offline
beginner

Joined: Sun Feb 26, 2006 3:16 am
Posts: 36
Has thanked: 0 time
Have thanks: 0 time
...


Last edited by Meds on Tue Apr 25, 2006 8:06 pm, edited 1 time in total.

Top
 Profile  
 
 Post subject:
PostPosted: Mon Mar 06, 2006 11:58 pm 
Offline
beginner

Joined: Sun Feb 26, 2006 3:16 am
Posts: 36
Has thanked: 0 time
Have thanks: 0 time
...


Last edited by Meds on Tue Apr 25, 2006 8:07 pm, edited 2 times in total.

Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 516 posts ]  Go to page 1, 2, 3, 4, 5 ... 35  Next


You can make the ads go away by registering


All times are UTC + 1 hour


Who is online

Users browsing this forum: No registered users and 7 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group