AC Rouge-Liberation HD (forge) Tools [PC]

[michalss]

Here is ARC and ACLHD repacker for all, sharing my tools, just a few things berfore you gonna use it. Repacker works only for non extended files and PC files!!! It is very complicated format and this will take time, so please be patients. Repack can be done only without the compression because i did not finish the crc adler check yet.

I have test repacker on all forge files and they are all working ok but with this ext files exception... If you like it please hit thanx button on the bottom..




Download Michalss AC Repacker

Code: Select all

https://own.cloudproject.cz/cloudproject/index.php/s/NoKtvUoc95VivRG

[MichaelDarkAngel]

Aside from the endianess, I can't see why this would be much different from AC:Unity or possibly even AC:4. As I haven't written any of the structures out since AC:1 this will take me as little longer than one sitting. So here's the start, and I'll keep filling it in.

Code: Select all

Location    Type        Notes
       0    Byte[8]     "scimitar" null-terminated string (The same in every AC/POP game, so far)
      13    Int64e      Index Table Pointer (absolute)

...Skip to Index Table Pointer (this file 1050)

    1050    Int32e      File Count (files contained in this forge file)
    1078    Int32e      Index File Count (files contained in this index)
    1102    Int64e      Index Start Pointer (absolute)
    1110    Int64e      Next Index Table Pointer (absolute, -1 if no other indexes are present)
    1118    Int32e      Start Index
    1122    Int32e      End Index
    1126    Int64e      Filename Table Pointer (absolute)
    1134    Int64e      End of Filename Table (absolute, possible pointer for something else unknown)

[IF multiple Index Tables are present] (repeat until no other indexes are present)

...Skip to Next Index Table Pointer (not present in this file, information based on ACIV:Black Flag forge file)

       0    Int32e      Index File Count (files contained in this index)
       8    Int64e      Index Start Pointer (absolute)
      16    Int64e      Next Index Table Pointer (absolute, -1 if no other indexes are present)
      24    Int32e      Start Index
      28    Int32e      End Index
      32    Int64e      Filename Table Pointer (absolute)
      40    Int64e      End of Filename Table (absolute, possible pointer for something unknown)

[ENDIF]

...Skip to Index Start Pointer (this file 1142)

Index block (repeat equal to File Count)

       0    Int64e      File Data Pointer (absolute)
       8    Int64e      File Data ID
      16    Int32e      File Data Size

...Skip to Filename Table Pointer (this file 1242)

Filename block (repeat equal to File Count)

       0    Int32e      File Data Size (should match file data size from index block respectively)
       4    Int64e      File ID
      40    Int32e      Timestamp
      44    Byte[128]   Filename null-terminated string (some filenames are empty strings)
     172    Byte[20]    Unknown

...Skip to first File Data Pointer

A couple edits to the above for clarification.

I haven't gotten much farther than this yet, and will continue tomorrow. So far, everything I have outlined has been little-endian. I have seen this with other xBox AC offerings so it comes as no surprise to me. That's why I have tagged the types above with a little "e", because that's how they should be read.

--MDA

[michalss]

man that is perfect... please continue with this and keep us posted...

thx

[michalss]

ok made a binary template for this and made some modigications also you have a small mistake in struct :

In line with 8 Int32e File Data ID is wrong need to be Int64e

also there is some kind of crc, do you anything about it pls ?



But i have to say your struct template made my day please if you can continue with this coz this are very rare informations

[MichaelDarkAngel]

ok made a binary template for this and made some modigications also you have a small mistake in struct :

In line with 8 Int32e File Data ID is wrong need to be Int64e

I see that now and fixed the post above, also had another error with a skip to. I wonder if this may have changed with Rogue and is the reason why my Unity routines won't open the files. Just to clarify, for anyone not following, I am referring to an issue loriscangini is having with the PC version of Rogue.

Actually, just checked a Unity forge file again and it looks like that should be 64-bit also. I'll have to see if that changes anything. Good eye michalss

also there is some kind of crc, do you anything about it pls ?

Not sure what you are referring to. Where at in the file?

Continuing on...

Next we tackle the individual "datafiles". Some of these will be compressed, some will not. This is also where things switch to big-endian.

We'll start with the compressed datafiles (3 out of the 4 datafiles in this forge file are compressed):

Code: Select all

Location    Type        Notes
       0    Int64E      17,179,869,184 - Not sure what this, not always present in the PC version
       8    Int64E      1,154,322,941,026,740,787 - This signifies a compressed chunk, may be more than one per datafile
      16    Int16E      Compression Type    1 = lzo1x
                                            2 = lzo2a
                                            5 = lzo1c
      18    Byte        Unknown
      19    UInt16E     Min Size (appears to always be 32,768)
      21    UInt16E     Max Size (appears to always be 32,768)
      23    UInt16E     LZO Block Count

LZO Blocks (repeat equal to LZO Block Count)

       0    UInt16E     Uncompressed block size
       2    UInt16E     Compressed block size

Compressed data (repeat equal to LZO Block Count)

       0    Int32E     Possible checksum
       4    Byte[xx]    Compressed data xx = Compressed block size for each respective LZO Block

At this point, if you are not beyond the file data size I would read another UInt64E. If it matches the compressed chunk signature, repeat the decompression routine. ARchive_neXt decompresses all chunks to a single file (original AC:1 information stated that wasn't necessary, but I find it easier to read if it's a single uncompressed file). If uncompressed size equals compressed size, there is no need to decompress that data. Also not positive all three of those compression types are valid for forge files, but they are the ones that I use in my dll. However, I believe you will only find that one type of compression is used throughout the game (in this case it should be lzo1x).

Uncompressed "datafiles":

Without looking at more all I can tell you is if the first UInt64E does not equal 17,179,869,184 or 1,154,322,941,026,740,787 back up 8 bytes and read it as uncompressed data equal to the file data size for the respective datafile.

For this forge file, the first datafile located at 3083 is not compressed, so starting at 3083 read 280 bytes and that would be the contents of "GlobalMetaFile". I have never figured out what these files are, every forge file has one.

Hope this helps somewhat. Next up I'll tackle reading the expanded datafiles.

--MDA

[michalss]

Hi,

I did another a few test to verify your finding and i have a few questions

Whole this part i guess is not right

Code: Select all

LZO Blocks (repeat equal to LZO Block Count)

       0    UInt16E     Uncompressed block size
       2    UInt16E     Compressed block size

Compressed data (repeat equal to LZO Block Count)

       0    Int32E     Possible checksum
       4    Byte[xx]    Compressed data xx = Compressed block size for each respective LZO Block

because it seems that LZO Block Count is always 1, and if maximum chunk is 32768 it means it must be more then only 1 Block Count Also it looks like kind of double header, strange there need to be some kind of how many chunks that file contains right ?
Can you please double check this ? Also this :

0 Int32E Possible checksum is actually 2 crc checks seems splitted in to int16 crc1 and int16 crc2 adler32 checksum... (infromation from other Ac games, not sure if true )


also uploaded bigger file:

Code: Select all

https://dl.dropboxusercontent.com/u/38234344/Data360.forge

[Gh0stBlade]

Anyone managed to extract the audio in this game? If so, what's the process of it?

Thanks.

[michalss]

Ok every byte from dat files is know now, thanks to my old templates and my one good friend( with crc part crap ) That crc part is most likely not really crc, but current position checksum calculation

Template is still a bit mess but it is very usable .... Thx MDA for help, now im able to write repacker for X360,PC version....

Code: Select all

//--------------------------------------
//--- 010 Editor v6.0.1 Binary Template
//
// File: ACr. DAT
// Author: michalss
// Revision: V2.0
// Purpose: X360 version of Game
//--------------------------------------

BigEndian();

local uint x,y,chunk_count,gamma,i,crckonew,crckoorig;

uint64 notsure;

struct HEAD_one {
uint64 sign;
ushort com_type;
byte lzo_block_ver;
ushort compress_size_block_max;
ushort uncompress_size_block_max;
ushort lzo_block_count;

local uint chunks_compress[lzo_block_count];
local uint chunks_deccompress[lzo_block_count];

for (x=0;x<lzo_block_count;x++) 
{
    struct COMP {
         ushort decompresschunksize;
         ushort compresschunksize;
         chunks_compress[y]=compresschunksize;
         chunks_deccompress[y]=decompresschunksize;
    } block;  
}

struct CRC_HedaerdataBlock {
for(i=0;i<lzo_block_count;i++) {
    struct blokdata_header {
        ushort crc1; // velikost+short, ze zabaleneho
        ushort crc2; // ADLER32+1
        gamma=(uint)((int)(crc1+chunks_compress[i]));
        do {
            if(gamma>65521) gamma=gamma-65521;
        } while(gamma>65521);
        crckonew=gamma*65536+crc2+1;
        crckoorig=Checksum(CHECKSUM_ADLER32,FTell(),chunks_compress[i]);
        if(crckoorig!=crckonew) {
            Printf("Problem CRC - Data %i: ",i);
            Printf("Counted: %X, ",crckonew);
            Printf("Original: %X \n",crckoorig);
        }
        ubyte data_chunk[chunks_compress[i]];
    } record;
}
} record;

} DATA;


struct HEAD_Chunks {

uint64 sign2;
ushort com_type2;
byte lzo_block_ver2;

ushort compress_size_block_max2;
ushort uncompress_size_block_max2;
ushort lzo_block_count2;
chunk_count=lzo_block_count2;

local uint chunks_compress[lzo_block_count2];
local uint chunks_deccompress[lzo_block_count2];

for (y=0;y<lzo_block_count2;y++) 
{
   struct UNKcomp { 

    ushort decompresschunksize;
    ushort compresschunksize;
    chunks_compress[y]=compresschunksize;
    chunks_deccompress[y]=decompresschunksize;

  } data;  
}

struct CRCdataBlock {
for(i=0;i<chunk_count;i++) {
    struct blokdata {
        ushort crc1; // velikost+short, ze zabaleneho
        ushort crc2; // ADLER32+1
        gamma=(uint)((int)(crc1+chunks_compress[i]));
        do {
            if(gamma>65521) gamma=gamma-65521;
        } while(gamma>65521);
        crckonew=gamma*65536+crc2+1;
        crckoorig=Checksum(CHECKSUM_ADLER32,FTell(),chunks_compress[i]);
        if(crckoorig!=crckonew) {
            Printf("Problem CRC - Data %i: ",i);
            Printf("Counted: %X, ",crckonew);
            Printf("Original: %X \n",crckoorig);
        }
        ubyte data_chunk[chunks_compress[i]];
    } record;
}
} record;

} DATA;

[michalss]

also 1 more think about compression: Im not 100% sure coz i dont have PC version yet but it looks like this is coorect...

Code: Select all

1 = xmemcompress (lzx)
2 = lzo2a
5 = lzo1c

[Mr.Mouse]

Great work there!

[michalss]

Great work there!

Thx this is very complicated format for me but it gave me a lot of new ideas

[MichaelDarkAngel]

Sorry for the delay. I guess I didn't explain the multi-chunk format well enough, but it looks like you figured it out on your own. Should I continue?

Have you attempted any re-packing yet? Would be interested in reading how it goes.

--MDA

[michalss]

Please continue with it.. I'll gonna work on repacker tommorow... I'll share my work once it is completed if you want you can use my template as reference it is completed include the Adler calculations...

[michalss]

I just wrote unpacker for scimitar files, however for repack purposes i need more information about this format, things like Lost and found and etc....

[michalss]

OK i finnish the repacker for scimitar files, not handling compressed chunks yet... My next work would be to make decompressor for compressed files and the repacker for them, this will take some time since they are chunked...