XeNTaX Forum Index
Forum MultiEx Commander Tools Tools Home
It is currently Sun May 27, 2018 4:35 am

All times are UTC + 1 hour


Forum rules


Please click here to view the forum rules



Post new topic Reply to topic  [ 34 posts ]  Go to page 1, 2, 3  Next
Author Message
 Post subject: Black Desert Online File Formats (PAB, PAC, PAA)
PostPosted: Wed Aug 27, 2014 3:28 am 
Offline
ultra-n00b

Joined: Wed Aug 27, 2014 3:24 am
Posts: 3
Has thanked: 2 times
Have thanks: 7 times
Update:
I believe I've cracked pretty much all of it just need to determine the format of the keyframe values and we are done.
Explaination of converting the Animation's compressed scale, rotation & position can be found in my other post here.

I feel I've almost figured out the details for all of these formats but I may need some input for the last stretch. Huge credits go to Ekey for his PAZUnpacker_CBT2 (without which I'd have nothing to do!), and to chrrox for their Noesis script (which is pretty much the foundation of what I've got). There are still a few things that aren't quite complete, primarily with the PAA files which I believe have some form of compression which I talk about in their section any input here would be awesome.

For those not familiar with the files:
  1. PAC - This file represents the model format (chrrox mapped most of this out in their script)
  2. PAB - This file represents the skeleton format which various models depend on (chrrox mapped this out in their script)
  3. PAA - This file represents the animation format, one file per animation

Common Structures
Below are some of the common structures in each of these files, listing them here will save some space and typing. They are pretty self explainatory. If anyone knows more about the unknown data in the FileHeader I'd be interested to know what it is, it hasn't seemed to be required so I haven't really looked at it.
Code:
typedef struct FileHeader
{
   char                fileId[4];
   unsigned_int16      fileVersion;

   unsigned_int8       unknown[10];
} FileHeader;

typedef struct Matrix4x4
{
   float       data[4][4];
} Matrix4x4;

typedef struct Vector3
{
   float      x;
   float      y;
   float      z;
} Vector3;

typedef struct Quaternion
{
   float      x;
   float      y;
   float      z;
   float      w;
} Quaternion;

typedef struct CompressedVector3
{
   int16      x;
   int16      y;
   int16      z;
} CompressedVector3;

typedef struct CompressedQuaternion
{
   int16      x;
   int16      y;
   int16      z;
   int16      w;
} CompressedQuaternion;


PAB File Format
This file starts with the standard FileHeader structure with the main data starting at 0x00000010 which lists the number of bones in the file and is followed by the array of bones.
I believe the unknown data is just a padding or chunk end or something, either way it doesn't seem to be required.
Once again, credit to chrrox who's Noesis script is the framework for this.
Code:
typedef struct Bone
{
   unsigned_int32      boneHash;
   unsigned_int8       boneNameLength;
   char                boneName[boneNameLength];
   unsigned_int32      boneParent;

   Matrix4x4           boneMatrix;
   Matrix4x4           boneMatrixInverse;
   Matrix4x4           boneLocalMatrix;
   Matrix4x4           boneLocalMatrixInverse;
   Vector3             boneScale;
   Quaternion          boneRotation;
   Vector3             bonePosition;

   unsigned_int8       unknown[2];
} Bone;

struct BoneFileFormat
{
   FileHeader       fileHeader;

   unsigned_int16   boneCount;
   Bone             bones[boneCount];
};


PAC File Format
Moving up in complexity slightly, this file has the mesh data. Each mesh has a texture name attached to it, I believe this is a base name so textures with other semantics (like normal map, ambient occlusion, specular power, etc) can be automatically added by searching for their relevant semantic id which is appended to the base texture name. I haven't really looked into determining what PAB file is supposed to be used for these yet, if it's not in the file then it may be a matter of searching the directory (and parent directories with a depth restriction) to find a PAB file which has the bone hashes that the PAC file contains. I've only included the version 259 of this, I haven't seen where the 513 version chrrox also uses is used (but there are a lot of models so easily possible I've missed it!)
Once again, credit to chrrox who's Noesis script is the framework for this.
Code:
typedef struct VertexBuffer
{
   Vector3            position;
   unsigned_int8      normals[4];         // Commented out by chrrox
   unsigned_int16     uv[2];
   unsigned_int8      colour[4];         // Commented out by chrrox
   unsigned_int8      boneIndices[4];
   unsigned_int8      boneWeights[4];
} VertexBuffer;


typedef struct LOD
{
   unsigned_int16         vertexCount;
   VertexBuffer           vertexBuffer[vertexCount];
   unsigned_int32         faceCount;
   unsigned_int16         faceIndices[faceCount];
} LOD;

typedef struct Mesh
{
   unsigned_int8          textureNameLength;
   char                   textureName[textureNameLength];
   unsigned_int16         flag;

   LOD                    lodArray[3];
} Mesh;

struct ModelFile
{
   FileHeader             fileHeader;

   unsigned_int8          boneCount;
   unsigned_int32         boneHashArray[boneCount];
   unsigned_int8          usedBoneCount;
   int8                   usedBoneArray[usedBoneCount];

   unsigned_int32         totalVertices;
   unsigned_int32         totalFaces;
   unsigned_int16         meshCount;

   Mesh                   meshes[meshCount];
};


PAA File Format
This file was a pain, it's easy knowing what data needs to be in an animation file yet people find so many different ways to do so. I believe there is some form of compression being used within this file for the scale, rotation and position (and possibly key frame / frame time).
The file starts the same as normal with the FileHeader followed by a count which for this file represents the number of bones in the animation, following this is 8 bytes of data (first 4 I believe are a float for the total animation time) and from here the bone animation data starts.
The animation data appears to compress all of the floats (e.g. time frame, scale, rotation and position components) into half's. I'm not so good at decryption so maybe someone can help out, at the least we can logically infer from the scale values that "E8 03" should equal 1.0F. Once again I such ad decompression so I'd love some input here.
With this knowledge the format for the animation file is as follows:
Code:
typedef struct ScaleData
{
   half                  keyframe;
   CompressedVector3    keyframeScale;  // Multiply values by 0.001F to get float value
} ScaleData;

typedef struct RotationData
{
   half                   keyframe;
   CompressedQuaternion   keyframeRotation;  // Multiply values by max value of their sign to get float value
                                                                  // value == 0 : (0.0F)
                                                                  // value < 0 : (value * 32768.0F)
                                                                  // value > 0 : (value * 32767.0F)
} RotationData;

typedef struct PositionData
{
   half                  keyframe;
   CompressedVector3    keyframePosition;  // Multiply values by 0.1F to get float value
} PositionData;

typedef struct AnimBoneData
{
   unsigned_int32      boneHash;

   unsigned_int16      scaleKeyframeCount;
   ScaleData           scaleData[scaleKeyframeCount];

   unsigned_int16      rotationKeyframeCount;
   RotationData        rotationData[rotationKeyframeCount];

   unsigned_int16      positionKeyframeCount;
   PositionData        positionData[positionKeyframeCount];
} BoneAnimationData;

struct Animation
{
   FileHeader          fileHeader;
   unsigned_int16      boneCount;

   float               animationDuration;   // ?
   unsigned_int8       unknown[4];          // ?

   AnimBoneData           boneData[boneCount];
};


Last edited by ArthainBaka on Wed Aug 27, 2014 10:25 am, edited 2 times in total.

Top
 Profile  
 
 Post subject: Re: Black Desert Online File Formats (PAB, PAC, PAA)
PostPosted: Wed Aug 27, 2014 6:18 am 
Offline
ultra-n00b
User avatar

Joined: Wed Oct 30, 2013 3:49 pm
Posts: 5
Has thanked: 2 times
Have thanks: 0 time
You forget .pam (object files)
i think all our answers in \object\parc.exe
help for parc.exe, getting in korean windows
Quote:
Pearl-Abyss Resource Compiler Ver1.0 01:11:13 08/25/14

Pearl-Abyss RC Command-line Interface:
-h [--help] Help
-i [--input_file] arg input file name
-o [--output_file] arg output file name
-I [--input_files] Arg input file name list
Save -p [--output_path] arg the output file folder
-b [--base_path] top as a reference when creating sub-folder structure path arg
create sub-folder structure when writing files -k [--keep_subpath] in the output folder,
If the file has -w [--overwrite] overwrite the same name exists
-l [--log_output_list] logging output file list
compilation -a [--all] for all files
performed recursively for -r [--recursive] in the sub-folder
create a texture reference list -t [--texture_list] mesh file
-g [--retargetting_animation] for generating animation retargeting
-v [--verbosity] detail message output
Doing a -s [--single_thread] single-threaded
Perform compile only files --bone_only pb
-m [--skinned_mesh_only] only pc file compilation
Compiled by -e [--check_coexist] Raw data files are guys and pair
Verify that the coexistence
-c [--change_mode] arg object to change compiler (c followed by n (General)
r (r3m) c (Curly Religion) h (House) l (LOD)
Put the flag should generate a file for the flag
k (key frame Animation) m (morphing) p (pam) text
Reads the file.
Do other files created in r3m -x [--change_readfile] is not the Text
Change the deviation -u [--reduce_deviation] the arg object must
If -d [--delete_readfile] the object mode and delete the file read.




Default: PARC.exe -cp -a

Object vertex to reduce levels (less if you are using does anhal.)
-u + a ~ f a = 0.5 (-ua)
b = 1.0 (-ub)
c = 1.5 (-uc)
d = 2.0 (-ud)
e = 2.5 (-ue)
f = 3.0 (-uf)
If the change to all sub-folders, you can simply put the -r.


Top
 Profile  
 
 Post subject: Re: Black Desert Online File Formats (PAB, PAC, PAA)
PostPosted: Wed Aug 27, 2014 9:48 am 
Offline
ultra-n00b

Joined: Wed Aug 27, 2014 3:24 am
Posts: 3
Has thanked: 2 times
Have thanks: 7 times
BloodForce wrote:
You forget .pam (object files)
i think all our answers in \object\parc.exe
help for parc.exe, getting in korean windows


I had a look at it but for whatever reason my IDA chucks a fit when it tries to analyse the file and I can't get it to run for me (though maybe it's because I'm on an English install with Korean language pack). Either way I had no luck with it, or analysing the main .exe for the client.
I haven't really looked into the PAM files, so I'm not really sure what they are yet. I'm more interested in the model + animations. Once I've got those I might look at the PAM files.


Anyway I think I've cracked it!

Rotation Data
The rotation data it's being stored in four int16's and a bit of trickery to deserialize it into a float. Basically you just need to read the int16 value and and divide it by the largest value (for the given sign) to yeild the float data for the quaternion.
So if we look at some sample data:
Code:
// Sample PAA Format Quaternion Data
7E A5 82 5A 00 00 00 00

If we break this into the format we know it's in (an array of four int16's) we get the following:
Break this into an array of four int16's
Code:
Hex: [7E A5, 82 5A,  00 00, 00 00]
Decimal: [-23170, 23170, 0, 0]


Now we can convert it into float data using some simple math (my psuedo code always tends to be C++ for some reason >.>):
Code:
#define FMAX_INT16    32767.0F
#define FMIN_INT16    32768.0F     // Positive value to preserve sign on division

// Unpacks int16 value (of a quaternion component) from the .PAA format into a float
float UnpackValue(int16 value)
{
    if (value < 0)
    {
        return ((float)value / FMIN_INT16);
    }
    else if (value > 0)
    {
        return ((float)value / FMAX_INT16);
    }

    // If we get this far then the value has to be 0
    return (0.0F);
}


So if we apply this to the data we get the following (which now surprisingly looked like real quaternion values):
Code:
float: [-0.70709F, 0.70711F, 0,0F, 0.0F]


Scale Data
Every single file I've looked at has had the same E8 03 values, interestingly this nicely converts to an int16 as 1000.
Without spending time looking into it I'd say it's pretty safe to assume this is a fixed decimal point representation, so just mutliply by 0.001F (so people know we want a float, otherwise you could just divide by 1000) to get the data as a float (results with 1.0F which I think is obviously a good standard value for bone scale):
Code:
int16: [1000, 1000, 1000]
// Multiply all values by 0.001F to yield float
float: [1.0F, 1.0F, 1.0F]


Position Data
This is where I may be wrong but it seems to be correct for all the models I'm cross-referencing with positions of exported in Maya (standard PAB bone matrices used).
It seems this is just another fixed decimal point value, which is shifted by 1, so multiply the int16 by 0.1 or divide by 10 (outcome will be the same so personal preference really, probably safer dividing by 0.1 as then people know the output is a float and not a decimal which would ignore the decimal values)
From data of the Windmill model, in Maya the "Bone_VerticalAxis" has the following position [-1371.124, -0.001, -0.000], now we know it's from different bone data so really if we can get one value close to this and have the process work for the rest of the bones similarly we can pretty much assume it's correct (or good enough).
Black Desert seems to not use a world scale of 1 = 1m (unless this windmill is almost 1.4km high >.>) so given that it's less than 1 unit = 1m then the single decimal resolution isn't really that bad (I'm guessing they use 1 unit = 1mm).
Anyway, data and results this one like scale is pretty simple:
Code:
int16: [-13711, 0, 0]
// Multiply values by 0.1F for float
float: [-1371.1F, 0.0F, 0.0F]  // Pretty much identical to Maya target of [-1371.124, -0.001, -0.000]


So nice and easy!
That just leaves the keyframe values the be determined and we'll have a complete functional format. Hopefully I'll get enough time to finish it today ^.^


Last edited by ArthainBaka on Wed Aug 27, 2014 10:39 am, edited 1 time in total.

Top
 Profile  
 
 Post subject: Re: Black Desert Online File Formats (PAB, PAC, PAA)
PostPosted: Wed Aug 27, 2014 10:16 am 
Offline
n00b

Joined: Thu Oct 31, 2013 2:55 pm
Posts: 15
Has thanked: 5 times
Have thanks: 0 time
Just wow! You guys are incredible. If you will be able to unpack all those files - just no words.


Top
 Profile  
 
 Post subject: Re: Black Desert Online File Formats (PAB, PAC, PAA)
PostPosted: Wed Aug 27, 2014 10:38 am 
Offline
ultra-n00b

Joined: Wed Aug 27, 2014 3:24 am
Posts: 3
Has thanked: 2 times
Have thanks: 7 times
Rynage wrote:
Just wow! You guys are incredible. If you will be able to unpack all those files - just no words.

I think I only did the easy part!
Ekey was the one that made the unpacker which makes all of this possible, I can guarantee without it I wouldn't have even looked into this!
And chrrox provided enough of a framework with his Noesis script that I could figure out what I have so quickly.
I can not thank those two enough.

I do a lot of game development so I think that helped me a lot since I work with these structures on pretty much a daily basis, I've never tried to unpack any file formats before (actually I recall that I datamined Fury when it came out but the files were essentially plain text) so this was actually really really fun (and oddly relaxing).

I don't think I have enough time to get a working script with Noesis today though as I need to go out for the rest of the night shortly. Tomorrow should be plenty of time to learn enough Python (for a programmer I often wonder why I know almost nothing about Python) and enough of the Noesis API to get something working.


Top
 Profile  
 
 Post subject: Re: Black Desert Online File Formats (PAB, PAC, PAA)
PostPosted: Wed Aug 27, 2014 2:44 pm 
Offline
n00b

Joined: Thu Oct 31, 2013 2:55 pm
Posts: 15
Has thanked: 5 times
Have thanks: 0 time
ArthainBaka wrote:
Rynage wrote:
Just wow! You guys are incredible. If you will be able to unpack all those files - just no words.

I think I only did the easy part!
Ekey was the one that made the unpacker which makes all of this possible, I can guarantee without it I wouldn't have even looked into this!
And chrrox provided enough of a framework with his Noesis script that I could figure out what I have so quickly.
I can not thank those two enough.

I do a lot of game development so I think that helped me a lot since I work with these structures on pretty much a daily basis, I've never tried to unpack any file formats before (actually I recall that I datamined Fury when it came out but the files were essentially plain text) so this was actually really really fun (and oddly relaxing).

I don't think I have enough time to get a working script with Noesis today though as I need to go out for the rest of the night shortly. Tomorrow should be plenty of time to learn enough Python (for a programmer I often wonder why I know almost nothing about Python) and enough of the Noesis API to get something working.


Yeah I see your point. Anyway I was aiming on .PAA files.
For now, it's not possible to unpack this to work.


Top
 Profile  
 
 Post subject: Re: Black Desert Online File Formats (PAB, PAC, PAA)
PostPosted: Sat Jan 03, 2015 3:15 pm 
Offline
mega-veteran
mega-veteran

Joined: Wed May 05, 2010 8:21 pm
Posts: 278
Location: Poland Głogów
Has thanked: 16 times
Have thanks: 542 times
Hi

Here is an animation importer for models from this game.
It requires Blender version 249 and Python 26.
I used chrrox script for Noesis viewtopic.php?f=16&t=10909 and ArthainBaka file format research .

Steps:
1.unpack importer
2.run Blender249.blend
3.in Blender Text Window press alt+p and select:
-pac file - for importing a skinned mesh
-pab file - for importing a skeleton
-paa file - for importing an animation
-pam file - for importing static objects

Updated 2015-01-10:
-added pam importer

Updated 2015-01-06:
- added new animation importer


Attachments:


You do not have the required permissions to view the files attached to this post. Register to gain access.



Last edited by Szkaradek123 on Sat Jan 10, 2015 6:28 pm, edited 3 times in total.

Top
 Profile  
 
 Post subject: Re: Black Desert Online File Formats (PAB, PAC, PAA)
PostPosted: Sun Jan 04, 2015 5:00 pm 
Offline
mega-veteran
mega-veteran

Joined: Sun Apr 06, 2014 8:06 pm
Posts: 178
Has thanked: 216 times
Have thanks: 6 times
Szkaradek123 wrote:
Hi

Here is an animation importer for models from this game.
It requires Blender version 249 and Python 26.
I used chrrox script for Noesis viewtopic.php?f=16&t=10909 and ArthainBaka file format research .

Steps:
1.unpack importer
2.run Blender249.blend
3.in Blender Text Window press alt+p and select:
-pac file - for importing a skinned mesh
-pab file - for importing a skeleton
-paa file - for importing an animation
Hi Szkaradek123, my friend, the script runs perfectly, great work


Top
 Profile  
 
 Post subject: Re: Black Desert Online File Formats (PAB, PAC, PAA)
PostPosted: Thu Jan 08, 2015 3:40 am 
Offline
ultra-n00b

Joined: Wed Dec 30, 2009 5:41 pm
Posts: 1
Has thanked: 1 time
Have thanks: 0 time
animation import error? help me.. :(


Attachments:


You do not have the required permissions to view the files attached to this post. Register to gain access.



Top
 Profile  
 
 Post subject: Re: Black Desert Online File Formats (PAB, PAC, PAA)
PostPosted: Mon Jan 12, 2015 7:59 pm 
Offline
ultra-n00b

Joined: Sun Dec 11, 2011 10:07 pm
Posts: 8
Has thanked: 6 times
Have thanks: 0 time
Szkaradek123 wrote:
Hi

Here is an animation importer for models from this game.
It requires Blender version 249 and Python 26.
I used chrrox script for Noesis viewtopic.php?f=16&t=10909 and ArthainBaka file format research .

Steps:
1.unpack importer
2.run Blender249.blend
3.in Blender Text Window press alt+p and select:
-pac file - for importing a skinned mesh
-pab file - for importing a skeleton
-paa file - for importing an animation
-pam file - for importing static objects

Updated 2015-01-10:
-added pam importer

Updated 2015-01-06:
- added new animation importer


Thanks for the importer!!
is any way to get the correct bone names?


Top
 Profile  
 
 Post subject: Re: Black Desert Online File Formats (PAB, PAC, PAA)
PostPosted: Sun Apr 26, 2015 1:22 pm 
Offline
ultra-n00b

Joined: Sun Aug 31, 2014 9:08 am
Posts: 2
Has thanked: 0 time
Have thanks: 0 time
anyone know that where is the eyebrow and eye iris or pupil image files?
recently I ported the Valkyrie model from the Korean Black Desert Client into SFM and Gmod.
but actually I couldn't find the texture files of eyebrows, pupil or iris. so i used the eyebrow image file from the Vindictus.
but you know, it is little bit awkward. I found the other things like body or face tattoo, eye line make-up, preview image of customization but
damn.. there is no eye brow, iris .dds file! I think it should be inside of PAT files.
PAT = Peal abyss Texture, isn't it? lol if you guys can rip those PAT files too. it must be wonderful! and i will appreciate about that!


Top
 Profile  
 
 Post subject: Re: Black Desert Online File Formats (PAB, PAC, PAA)
PostPosted: Sun Apr 26, 2015 6:45 pm 
Offline
veteran

Joined: Sat Apr 19, 2014 8:02 am
Posts: 94
Has thanked: 44 times
Have thanks: 10 times
Would anyone that still has the BDO tools posted in viewtopic.php?f=10&t=10879 but later removed mind pm'ing me a download link? Not being able to extract the files makes it hard to use the tools posted here :D


Top
 Profile  
 
 Post subject: Re: Black Desert Online File Formats (PAB, PAC, PAA)
PostPosted: Wed May 06, 2015 11:34 am 
Offline
n00b
User avatar

Joined: Sun Feb 09, 2014 3:42 am
Posts: 11
Location: BeiJing China
Has thanked: 3 times
Have thanks: 0 time
@Szkaradek123 run step 3 alt+P just popup error window"python script error:check console"
:oops:


Top
 Profile  
 
 Post subject: Re: Black Desert Online File Formats (PAB, PAC, PAA)
PostPosted: Wed Feb 03, 2016 5:51 am 
Offline
ultra-n00b

Joined: Sun Jul 29, 2012 12:38 pm
Posts: 2
Has thanked: 0 time
Have thanks: 0 time
Thx for scripts and Tools .. Work perfect as import Model and import Animation ( 100 % ) in Blender 2.49b



Attachments:


You do not have the required permissions to view the files attached to this post. Register to gain access.



Top
 Profile  
 
 Post subject: Re: Black Desert Online File Formats (PAB, PAC, PAA)
PostPosted: Tue Mar 01, 2016 4:00 pm 
Offline
beginner

Joined: Thu Jul 30, 2015 5:05 am
Posts: 39
Has thanked: 13 times
Have thanks: 3 times
Do the tools used for extracting .paz exist anymore? if so, could someone please pm me?!


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 34 posts ]  Go to page 1, 2, 3  Next

All times are UTC + 1 hour


Who is online

Users browsing this forum: Google [Bot] and 6 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group