XeNTaX Forum Index
Forum MultiEx Commander Tools Tools Home
It is currently Sat Oct 21, 2017 3:31 pm

All times are UTC + 1 hour


Forum rules


Please click here to view the forum rules



Post new topic Reply to topic  [ 26 posts ]  Go to page 1, 2  Next
Author Message
 Post subject: Destiny (.pkg) XBOX360
PostPosted: Thu Sep 18, 2014 3:25 pm 
Offline
ultra-veteran
ultra-veteran
User avatar

Joined: Fri Apr 27, 2012 9:37 am
Posts: 430
Has thanked: 29 times
Have thanks: 178 times














You can make the ads go away by registering

I took a quick 5min look

Code:
//--------------------------------------
// File: .pkg (BIG Endian)
// Author: Cra0(Cra0kalo)
// Revision: 1
// Purpose: Research
//--------------------------------------

typedef struct
{
  uint32    magic;  //Always 00 18 00 03
  uint16    nElementCount;
  uint16    version;
  uint32    unknownA;
  uint32    unknownB;
  uint32    blankA;
  uint32    flagA;
  uint32    constValA; //doesnt change?
  uint32    constValB; //count maybe const
  byte      stampText[132];
} PK_Header;



//File Attribute Table (FAT) ???
typedef struct
{
  uint32    unknownA;
  uint32    unknownB;
  uint32    unknownC;
  uint32    dataPointer; //points to data
  uint32    unkParA;
  uint32    unkParB;
  byte      chunk[20];
} PK_FATEntryA;

typedef struct
{
  uint32    unknownA;
  uint32    dataPointer; //points to data
  byte      chunk[20];
} PK_FATEntryB;


BigEndian();
PK_Header header;
PK_FATEntryA entry1;
PK_FATEntryB entry2;
PK_FATEntryB entry3;


Not encrypted prob compressed with lzx

Samples
https://www.dropbox.com/sh/59pdrm5qn06l ... vlL4a?dl=0

PM me for xex

010 editor script >


Attachments:


You do not have the required permissions to view the files attached to this post. Register to gain access.


_________________
Devblog


Top
 Profile  
 
 Post subject: Re: Destiny (.pkg) XBOX360
PostPosted: Fri Sep 19, 2014 2:29 pm 
Offline
ultra-veteran
ultra-veteran

Joined: Thu Aug 12, 2010 8:15 am
Posts: 342
Has thanked: 54 times
Have thanks: 70 times
Here is a list of external tools that Bungie used in the game :)

Havok
CRIWARE
Speedtree
FaceFX
Umbra3 Visibility Solution
ZLIB Library
Freetype project
Granny Animation
Dolby Digital
Audiokinetic Wwise
Oodle


Top
 Profile  
 
 Post subject: Re: Destiny (.pkg) XBOX360
PostPosted: Sat Sep 20, 2014 4:39 am 
Offline
ultra-veteran
ultra-veteran
User avatar

Joined: Fri Apr 27, 2012 9:37 am
Posts: 430
Has thanked: 29 times
Have thanks: 178 times
brendan19 wrote:



yep here is the string dump of the executable
https://www.dropbox.com/s/h4ii6qpbfgsta ... p.zip?dl=0

_________________
Devblog


Top
 Profile  
 
 Post subject: Re: Destiny (.pkg) XBOX360
PostPosted: Wed Sep 24, 2014 9:26 am 
Offline
n00b

Joined: Sun Oct 04, 2009 5:37 pm
Posts: 10
Has thanked: 2 times
Have thanks: 11 times
The files seem to be compressed using LZHLW from the Oodle Compression Library (by RAD Game Tools [http://www.radgametools.com/oodle.htm]).

The file entries are:

Code:
struct DestinyFileEntry {
uint32_t FileOffset;
uint32_t CompressedFileSize;
uint32_t Flags; // 3rd byte tells the game whether the data is compressed
byte Sha1FileHash; // hash of the compressed or raw file data
};


The game also hashes the file tables and stores those hashes in the header as SHA-1's and the header is signed with SHA-256->RSA, and the signature is stored at 0x800.

If anyone has access to the Oodle SDK, I'm sure they can decompress them as there is no encryption on the data, just signed.

I'm not good with graphic formats or models, but this is an example of what a decompressed file looks like.
Image


Top
 Profile  
 
 Post subject: Re: Destiny (.pkg) XBOX360
PostPosted: Wed Sep 24, 2014 6:10 pm 
Offline
ultra-veteran
ultra-veteran
User avatar

Joined: Fri Apr 27, 2012 9:37 am
Posts: 430
Has thanked: 29 times
Have thanks: 178 times
Patriot wrote:
The files seem to be compressed using LZHLW from the Oodle Compression Library (by RAD Game Tools [http://www.radgametools.com/oodle.htm]).

The file entries are:

Code:
struct DestinyFileEntry {
uint32_t FileOffset;
uint32_t CompressedFileSize;
uint32_t Flags; // 3rd byte tells the game whether the data is compressed
byte Sha1FileHash; // hash of the compressed or raw file data
};


The game also hashes the file tables and stores those hashes in the header as SHA-1's and the header is signed with SHA-256->RSA, and the signature is stored at 0x800.

If anyone has access to the Oodle SDK, I'm sure they can decompress them as there is no encryption on the data, just signed.

I'm not good with graphic formats or models, but this is an example of what a decompressed file looks like.
Image


great work ill look into it some more soon is there by any chance you can get some sample uncompressed files?

_________________
Devblog


Top
 Profile  
 
 Post subject: Re: Destiny (.pkg) XBOX360
PostPosted: Fri Sep 26, 2014 8:44 pm 
Offline
mega-veteran
mega-veteran

Joined: Thu Apr 17, 2008 3:48 am
Posts: 278
Has thanked: 47 times
Have thanks: 38 times
cra0 wrote:
Patriot wrote:
The files seem to be compressed using LZHLW from the Oodle Compression Library (by RAD Game Tools [http://www.radgametools.com/oodle.htm]).

The file entries are:

Code:
struct DestinyFileEntry {
uint32_t FileOffset;
uint32_t CompressedFileSize;
uint32_t Flags; // 3rd byte tells the game whether the data is compressed
byte Sha1FileHash; // hash of the compressed or raw file data
};


The game also hashes the file tables and stores those hashes in the header as SHA-1's and the header is signed with SHA-256->RSA, and the signature is stored at 0x800.

If anyone has access to the Oodle SDK, I'm sure they can decompress them as there is no encryption on the data, just signed.

I'm not good with graphic formats or models, but this is an example of what a decompressed file looks like.
Image


great work ill look into it some more soon is there by any chance you can get some sample uncompressed files?
4

Oodle has five primary compression formats btw, LZA probably isnt the one choosen (since it was made reasonable speed-wise for games in july, so it's unlikely it was implemented into destiny) LZNib http://cbloomrants.blogspot.com/2012/09 ... lznib.html is a possibility, other hints as to the structures are on this blog. he seems to be a dev for rad game tools. Also looks like alot of other next-gen games are picking up Oodle for use, The Order: 1886 is supposedly using it also (uncomfirmed, found through forum posts on gamedev.net by a programmer who works for ready at dawn studios)


Top
 Profile  
 
 Post subject: Re: Destiny (.pkg) XBOX360
PostPosted: Sat Sep 27, 2014 9:57 am 
Offline
ultra-n00b

Joined: Sat Sep 27, 2014 9:52 am
Posts: 3
Has thanked: 0 time
Have thanks: 0 time
I'm willing to dedicate a lot of time to this project as I'm fairly invested in learning how to unpack game files. Is there anyways as a scrub that I can contribute, or should I just go learn elsewhere?


Top
 Profile  
 
 Post subject: Re: Destiny (.pkg) XBOX360
PostPosted: Sat Sep 27, 2014 12:01 pm 
Offline
ultra-veteran
ultra-veteran
User avatar

Joined: Fri Apr 27, 2012 9:37 am
Posts: 430
Has thanked: 29 times
Have thanks: 178 times
I tried legitimately to contact someone from radgametools on getting my hands on the oodle package to evaluate however lets just say their licensing price is up there in the thousands.

Anyway
Image

Crude as hell but here
http://cra0kalo.com/public/DestinyPKGTool.zip

_________________
Devblog


Top
 Profile  
 
 Post subject: Re: Destiny (.pkg) XBOX360
PostPosted: Sat Sep 27, 2014 4:04 pm 
Offline
n00b

Joined: Sun Oct 04, 2009 5:37 pm
Posts: 10
Has thanked: 2 times
Have thanks: 11 times
Pepper wrote:
4

Oodle has five primary compression formats btw, LZA probably isnt the one choosen (since it was made reasonable speed-wise for games in july, so it's unlikely it was implemented into destiny) LZNib http://cbloomrants.blogspot.com/2012/09 ... lznib.html is a possibility, other hints as to the structures are on this blog. he seems to be a dev for rad game tools. Also looks like alot of other next-gen games are picking up Oodle for use, The Order: 1886 is supposedly using it also (uncomfirmed, found through forum posts on gamedev.net by a programmer who works for ready at dawn studios)


I'm 99% (1% not sure since I don't have their SDK) certain it's LZHLW since that's what I found from looking at the function that decompresses the files in the game.

Also, I can make decompressed file dumps later this upcoming week. I have exams and after that I can create something to dump all files that run through the decompressor.


Top
 Profile  
 
 Post subject: Re: Destiny (.pkg) XBOX360
PostPosted: Sat Sep 27, 2014 9:59 pm 
Offline
mega-veteran
mega-veteran

Joined: Thu Apr 17, 2008 3:48 am
Posts: 278
Has thanked: 47 times
Have thanks: 38 times
Patriot wrote:
Pepper wrote:
4

Oodle has five primary compression formats btw, LZA probably isnt the one choosen (since it was made reasonable speed-wise for games in july, so it's unlikely it was implemented into destiny) LZNib http://cbloomrants.blogspot.com/2012/09 ... lznib.html is a possibility, other hints as to the structures are on this blog. he seems to be a dev for rad game tools. Also looks like alot of other next-gen games are picking up Oodle for use, The Order: 1886 is supposedly using it also (uncomfirmed, found through forum posts on gamedev.net by a programmer who works for ready at dawn studios)


I'm 99% (1% not sure since I don't have their SDK) certain it's LZHLW since that's what I found from looking at the function that decompresses the files in the game.

Also, I can make decompressed file dumps later this upcoming week. I have exams and after that I can create something to dump all files that run through the decompressor.



No problem, I didn't have any reason to suspect LZNib, I just wanted to give any information I could find online.


Top
 Profile  
 
 Post subject: Re: Destiny (.pkg) XBOX360
PostPosted: Fri Oct 03, 2014 9:21 pm 
Offline
n00b

Joined: Sun Oct 04, 2009 5:37 pm
Posts: 10
Has thanked: 2 times
Have thanks: 11 times
Here's the first file from "360_client_bootstrap_unpatchable_0.pkg". Will dump more in the weekend, trying to setup a tool to just feed the game any compressed file but I have to figure out the Oodle decompression function's parameters first.

I decided to upload this one since it's the first file that is decompressed.

This file contains the four compressed fragments in the above file and one "full" file, which is just the four fragments stitched together. The rest of the file is not compressed so you can just extract the binary data.

http://www.mediafire.com/download/5nzicjr2s5zwwkl


Update on research:

The resource entry table (offset of table is at 0xB8 in the pkg files) contains positions, sizes and indexes of the resources the game access. Each compressed file decompresses to 0x40000 bytes and all of the fragments decompressed make up a file.

The positions in each resource entry is relative to the start of the decompressed chunk. So for example, if accessing entry 0x1B0, the position value could be something like 0x00000040 instead of (0x1B0 * 0x40000) + 0x00000040.

Also, sound files don't seem to be compressed, you can extract them and they are in the XMA format, so to play them as .wav on pc, they need to be converted.


Update on files:

I can decompress any of the files now, if you want to do research on the files contact me through Private Message and we can work on them.


Top
 Profile  
 
 Post subject: Re: Destiny (.pkg) XBOX360
PostPosted: Thu Oct 09, 2014 6:49 am 
Offline
mega-veteran
mega-veteran

Joined: Sun Apr 06, 2014 8:06 pm
Posts: 178
Has thanked: 214 times
Have thanks: 6 times
Patriot wrote:
Here's the first file from "360_client_bootstrap_unpatchable_0.pkg". Will dump more in the weekend, trying to setup a tool to just feed the game any compressed file but I have to figure out the Oodle decompression function's parameters first.

I decided to upload this one since it's the first file that is decompressed.

This file contains the four compressed fragments in the above file and one "full" file, which is just the four fragments stitched together. The rest of the file is not compressed so you can just extract the binary data.

http://www.mediafire.com/download/5nzicjr2s5zwwkl


Update on research:

The resource entry table (offset of table is at 0xB8 in the pkg files) contains positions, sizes and indexes of the resources the game access. Each compressed file decompresses to 0x40000 bytes and all of the fragments decompressed make up a file.

The positions in each resource entry is relative to the start of the decompressed chunk. So for example, if accessing entry 0x1B0, the position value could be something like 0x00000040 instead of (0x1B0 * 0x40000) + 0x00000040.

Also, sound files don't seem to be compressed, you can extract them and they are in the XMA format, so to play them as .wav on pc, they need to be converted.


Update on files:

I can decompress any of the files now, if you want to do research on the files contact me through Private Message and we can work on them.
Hello, my friends, thank you very much for your great work, I want to like you, wanted to develop better tools to share with you, sharing happy, thank you


Top
 Profile  
 
 Post subject: Re: Destiny (.pkg) XBOX360
PostPosted: Thu Oct 09, 2014 7:14 am 
Offline
ultra-veteran
ultra-veteran
User avatar

Joined: Fri Apr 27, 2012 9:37 am
Posts: 430
Has thanked: 29 times
Have thanks: 178 times
Patriot wrote:
Here's the first file from "360_client_bootstrap_unpatchable_0.pkg". Will dump more in the weekend, trying to setup a tool to just feed the game any compressed file but I have to figure out the Oodle decompression function's parameters first.

I decided to upload this one since it's the first file that is decompressed.

This file contains the four compressed fragments in the above file and one "full" file, which is just the four fragments stitched together. The rest of the file is not compressed so you can just extract the binary data.

http://www.mediafire.com/download/5nzicjr2s5zwwkl


Update on research:

The resource entry table (offset of table is at 0xB8 in the pkg files) contains positions, sizes and indexes of the resources the game access. Each compressed file decompresses to 0x40000 bytes and all of the fragments decompressed make up a file.

The positions in each resource entry is relative to the start of the decompressed chunk. So for example, if accessing entry 0x1B0, the position value could be something like 0x00000040 instead of (0x1B0 * 0x40000) + 0x00000040.

Also, sound files don't seem to be compressed, you can extract them and they are in the XMA format, so to play them as .wav on pc, they need to be converted.


Update on files:

I can decompress any of the files now, if you want to do research on the files contact me through Private Message and we can work on them.


Could you update this?
https://github.com/cra0kalo/DestinyPKGT ... ool/PKG.cs

_________________
Devblog


Top
 Profile  
 
 Post subject: Re: Destiny (.pkg) XBOX360
PostPosted: Thu Oct 09, 2014 7:19 am 
Offline
n00b

Joined: Sun Oct 04, 2009 5:37 pm
Posts: 10
Has thanked: 2 times
Have thanks: 11 times
cra0 wrote:


Yea, I'll try to fill in as much as I know.

Also, I was able to extract a texture:

http://gyazo.com/f46ca672f9aeb9b40637c00d79231fb5

Still working on dealing with graphics formats, injection/editing is some time away.


Top
 Profile  
 
 Post subject: Re: Destiny (.pkg) XBOX360
PostPosted: Thu Oct 09, 2014 1:35 pm 
Offline
ultra-veteran
ultra-veteran

Joined: Thu Aug 12, 2010 8:15 am
Posts: 342
Has thanked: 54 times
Have thanks: 70 times
Is there a way get the TOC from the PKG files so we can get some filenames going or are we going to be stuck with hashes?


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 26 posts ]  Go to page 1, 2  Next

All times are UTC + 1 hour


Who is online

Users browsing this forum: awesomenessalal, Google [Bot], Yahoo [Bot] and 10 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group