XeNTaX Forum Index
Forum MultiEx Commander Tools Tools Home
It is currently Thu Dec 13, 2018 10:33 am

All times are UTC + 1 hour


Forum rules


Please click here to view the forum rules



Post new topic Reply to topic  [ 12 posts ] 
Author Message
 Post subject: text compression
PostPosted: Wed Dec 21, 2011 2:01 am 
Offline
advanced

Joined: Fri Sep 18, 2009 6:20 pm
Posts: 67
Has thanked: 16 times
Have thanks: 1 time

The contents of this post was deleted because of possible forum rules violation.

You can make the ads go away by registering



Top
 Profile  
 
 Post subject: Re: text compression
PostPosted: Wed Dec 21, 2011 2:52 am 
Offline
veteran

Joined: Tue Nov 27, 2007 8:30 pm
Posts: 94
Has thanked: 20 times
Have thanks: 5 times
This looks more like a in-game script than just text. It's partially compiled. I'm not sure of the format, but I do remember that Bounty Hunter was created using an in-house engine at Lucasarts.


Top
 Profile  
 
 Post subject: Re: text compression
PostPosted: Wed Dec 21, 2011 4:32 pm 
Offline
advanced

Joined: Fri Sep 18, 2009 6:20 pm
Posts: 67
Has thanked: 16 times
Have thanks: 1 time
Teancum wrote:
This looks more like a in-game script than just text. It's partially compiled. I'm not sure of the format, but I do remember that Bounty Hunter was created using an in-house engine at Lucasarts.

I don't think it is a script because in the archives directory the file name ends in .txt, and there are some strings that are cut off making me think it is compression.


Top
 Profile  
 
 Post subject: Re: text compression
PostPosted: Thu Dec 22, 2011 1:04 am 
Offline
mega-veteran
mega-veteran

Joined: Fri Apr 29, 2005 2:39 pm
Posts: 184
Has thanked: 5 times
Have thanks: 22 times
Looks LZSS


Top
 Profile  
 
 Post subject: Re: text compression
PostPosted: Thu Dec 22, 2011 2:00 am 
Offline
advanced

Joined: Fri Sep 18, 2009 6:20 pm
Posts: 67
Has thanked: 16 times
Have thanks: 1 time
Vash wrote:
Looks LZSS


I tried http://dev.gameres.com/Program/Other/LZSS.C as well as http://www.koders.com/c/fidC554142F5E42 ... 92DF8.aspx and neither decoded the file correctly.


Top
 Profile  
 
 Post subject: Re: text compression
PostPosted: Thu Dec 22, 2011 1:49 pm 
Offline
mega-veteran
mega-veteran

Joined: Fri Apr 29, 2005 2:39 pm
Posts: 184
Has thanked: 5 times
Have thanks: 22 times
yeah, seems a non-standard approach. Apparently instead of 1b flag 8b code uses 2b flag 16b code. With some strange variants...I'd need the uncompressed file to be able to crack it successfully. You can obtain it by looking for a compressed text file that you know appears in the beginning of the game than dump it from the ram/savestate. I don't know for which console is this game so I can't be more specific


Top
 Profile  
 
 Post subject: Re: text compression
PostPosted: Thu Dec 22, 2011 5:33 pm 
Offline
advanced

Joined: Fri Sep 18, 2009 6:20 pm
Posts: 67
Has thanked: 16 times
Have thanks: 1 time
Vash wrote:
yeah, seems a non-standard approach. Apparently instead of 1b flag 8b code uses 2b flag 16b code. With some strange variants...I'd need the uncompressed file to be able to crack it successfully. You can obtain it by looking for a compressed text file that you know appears in the beginning of the game than dump it from the ram/savestate. I don't know for which console is this game so I can't be more specific

It's from the PS2, and I don't have any methods of dumping the memory.

EDIT: Are there any magic values I could search in the ELF? I could re-write the decompression code in C from the MIPS assembly if I knew the function address.


Top
 Profile  
 
 Post subject: Re: text compression
PostPosted: Thu Dec 22, 2011 6:06 pm 
Offline
mega-veteran
mega-veteran

Joined: Fri Apr 29, 2005 2:39 pm
Posts: 184
Has thanked: 5 times
Have thanks: 22 times
Unfortunately not, it's pretty basic math functions...
You can dump it by making a savestate with the emulator when the text you've chosen appears on screen, then unpack the file with winrar/7zip or whatever and you'll have a file that weights 32MB (eememory.bin, but I'm not sure, can't check right now). That's the ram, now just look for the line of text inside there and you can cut the uncompressed file :)


Top
 Profile  
 
 Post subject: Re: text compression
PostPosted: Fri Dec 23, 2011 12:10 am 
Offline
advanced

Joined: Fri Sep 18, 2009 6:20 pm
Posts: 67
Has thanked: 16 times
Have thanks: 1 time
Vash wrote:
Unfortunately not, it's pretty basic math functions...
You can dump it by making a savestate with the emulator when the text you've chosen appears on screen, then unpack the file with winrar/7zip or whatever and you'll have a file that weights 32MB (eememory.bin, but I'm not sure, can't check right now). That's the ram, now just look for the line of text inside there and you can cut the uncompressed file :)


Thanks for your help, but I'm more interested in decompressing the textures. I just used a text file as an example since I figured it would be easier to analyze.


Top
 Profile  
 
 Post subject: Re: text compression
PostPosted: Fri Dec 23, 2011 10:06 am 
Offline
mega-veteran
mega-veteran

Joined: Fri Apr 29, 2005 2:39 pm
Posts: 184
Has thanked: 5 times
Have thanks: 22 times
yeah, and why wouldn't you extract some text to try to crack the compression? :oops:


Top
 Profile  
 
 Post subject: Re: text compression
PostPosted: Sat Jul 07, 2012 2:24 am 
Offline
advanced

Joined: Fri Sep 18, 2009 6:20 pm
Posts: 67
Has thanked: 16 times
Have thanks: 1 time
I've managed to extract some uncompressed data from memory. The bad part is that it isn't 100% text. The game decompresses text to the stack and parses it there, so I can't really find it in memory as it all happens too quickly. If someone knows how to debug games in the pcsx2 emulator I could extract the text data.

Here are the two files. One extracted from the archive and compressed, the other extracted from memory and uncompressed.

EDIT: files removed. If someone thinks they can help, please pm me. :bleh:


Top
 Profile  
 
 Post subject: Re: text compression
PostPosted: Sun Jun 15, 2014 10:14 pm 
Offline
advanced

Joined: Fri Sep 18, 2009 6:20 pm
Posts: 67
Has thanked: 16 times
Have thanks: 1 time
Almost exactly two years later, I have reverse engineered the decompression method. Here's the code:

Code:
#define BLOCK_SIZE (0x10)

typedef unsigned    char BYTE;
typedef unsigned    int  WORD; /* ps2 WORD = 4 bytes */
typedef signed       int    S_WORD; /* signed */

inline WORD read_marker(BYTE **src, WORD *block_index) {
   WORD marker;

   (*block_index) = BLOCK_SIZE;
   marker = (*src)[1] << 8 | (*src)[0];
   (*src) += 2;

   return marker;
}

inline WORD cycle_block(BYTE **src, WORD *block_index, WORD *marker) {
   WORD block_bit;

   block_bit = (*marker) & 1;
   (*block_index)--;

   if ((*block_index) > 0) {
      (*marker) >>= 1;
   } else {
      (*marker) = read_marker(src, block_index);
   }

   return block_bit;
}

WORD decompress(BYTE *src, BYTE *dest) {
   BYTE *dest_start, *dest_sub;
   WORD sub_index, sub_count;
   WORD marker, block_index;
   WORD bit_a, bit_b, byte_a, byte_b, byte_c; // dunno what these are called

   dest_start = dest;

   // ... read first marker in the file
   marker = read_marker(&src, &block_index);

PROCESS_MARKERS:
   // copy if regular data
   if (cycle_block(&src, &block_index, &marker) == 1) {
      dest++[0] = src++[0];
      goto PROCESS_MARKERS;
   }

   if (cycle_block(&src, &block_index, &marker) == 0) {
      bit_a = cycle_block(&src, &block_index, &marker) << 1;
      bit_b = cycle_block(&src, &block_index, &marker);

      sub_count = (bit_a | bit_b) + (WORD)3;
      sub_index = src++[0] | (WORD)0xFFFFFF00;

   } else {

      byte_a = src[1];
      byte_b = src[0];
      src += 2;

      sub_index = (((byte_a & 0xF0) << 4) - (WORD)0x1000) | byte_b;
      sub_count =     (byte_a & 0x0F) + 3;

      if (sub_count != 3) {
         goto SUB_COPY;
      } else if ((byte_c = src++[0]) != 0) {
         sub_count = (byte_c + 1);
         goto SUB_COPY;
      }

      // else we're done decompressing

      return (dest - dest_start); // total decompresed
   }

SUB_COPY:
   dest_sub = (dest + (S_WORD)sub_index);

   if (sub_count == 0) {
      sub_count--;
      goto PROCESS_MARKERS;
   } else {
      do {
         dest++[0] = dest_sub++[0];
      } while (--sub_count > 0);
   }

   goto PROCESS_MARKERS;

   return 0;
}


To the thousands of people who will use this: you're welcome.
:bye: :wmg: :bravo:



Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 12 posts ] 

All times are UTC + 1 hour


Who is online

Users browsing this forum: No registered users and 3 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group