XeNTaX
http://forum.xentax.com/

Deus EX: Mankind Divided .archive
http://forum.xentax.com/viewtopic.php?f=10&t=14928
Page 6 of 10

Author:  Vindis [ Sat May 06, 2017 1:48 pm ]
Post subject:  Re: Deus EX: Mankind Divided .archive

volfin wrote:
your command line is fine. But it would seem it's not finding any valid *.vap files in the 'input' folder. And just in case you think it will parse down subdirectories inside the 'input' folder, be assured it will not. It will only process *.vap files in 'input' folder itself. If your *.vap files are in input\someother directory, then you need to specify that directly.


Hi Volfin,

Tried to convert all txet (assuming those are the texture files) but I got an error, "unknown format -1" for each one. I renamed every file to vap.
I extracted the files with Sir Kanes' extractor, could be that problem? Should I try quickbms?

Thank you for your help!

Author:  volfin [ Sat May 06, 2017 5:06 pm ]
Post subject:  Re: Deus EX: Mankind Divided .archive

Vindis wrote:
volfin wrote:
your command line is fine. But it would seem it's not finding any valid *.vap files in the 'input' folder. And just in case you think it will parse down subdirectories inside the 'input' folder, be assured it will not. It will only process *.vap files in 'input' folder itself. If your *.vap files are in input\someother directory, then you need to specify that directly.


Hi Volfin,

Tried to convert all txet (assuming those are the texture files) but I got an error, "unknown format -1" for each one. I renamed every file to vap.
I extracted the files with Sir Kanes' extractor, could be that problem? Should I try quickbms?

Thank you for your help!


yes everything was written for the quickbms files. I don't know the differences involved with the other extractor.

Author:  Vindis [ Wed May 17, 2017 4:23 pm ]
Post subject:  Re: Deus EX: Mankind Divided .archive

volfin wrote:
Vindis wrote:
volfin wrote:
your command line is fine. But it would seem it's not finding any valid *.vap files in the 'input' folder. And just in case you think it will parse down subdirectories inside the 'input' folder, be assured it will not. It will only process *.vap files in 'input' folder itself. If your *.vap files are in input\someother directory, then you need to specify that directly.


Hi Volfin,

Tried to convert all txet (assuming those are the texture files) but I got an error, "unknown format -1" for each one. I renamed every file to vap.
I extracted the files with Sir Kanes' extractor, could be that problem? Should I try quickbms?

Thank you for your help!


yes everything was written for the quickbms files. I don't know the differences involved with the other extractor.


Is there any chance, that you share the file structure of the texture files, so I can write a program to work with Sir Kanes' extractor? Maybe there's not much difference...

Author:  volfin [ Wed May 17, 2017 5:13 pm ]
Post subject:  Re: Deus EX: Mankind Divided .archive

Vindis wrote:
Is there any chance, that you share the file structure of the texture files, so I can write a program to work with Sir Kanes' extractor? Maybe there's not much difference...


The header is as follows:

Code:
struct HeaderVAP
{
   long Unknown0;
   short Unknown1;
   short Unknown2;
   short Unknown7;
   short Unknown8;
   short Width;
   short Height;
   short Format1; // MIP count maybe?
   short Format;
   long Unknown3;
   long Unknown4;
   long Unknown5;
   long Unknown6;
};


Directly after the header comes the compressed texture data. How that data is formatted is dependent on the "Format" variable. The known "Format" codes are:

Code:
      TEXTURE AttemptedFormat = TEXTURE::NO_FORMAT;
      if (VAPHeader->Format == 0x4A)
         AttemptedFormat = TEXTURE::DXT1;
      else if (VAPHeader->Format == 0x50)
         AttemptedFormat = TEXTURE::DXT5;
      else if (VAPHeader->Format == 0x53)
         AttemptedFormat = TEXTURE::BC4;
      else if (VAPHeader->Format == 0x56)
         AttemptedFormat = TEXTURE::BC5;
      else if (VAPHeader->Format == 0x5B)
         AttemptedFormat = TEXTURE::BC7;
      else if (VAPHeader->Format == 0x43)
         AttemptedFormat = TEXTURE::U8bpp;
      else if (VAPHeader->Format == 0x1D)
         AttemptedFormat = TEXTURE::U32bpp;

      if (AttemptedFormat == TEXTURE::NO_FORMAT)
      {
         printf("Unknown Format %d\n", VAPHeader->Format);
         continue;
      }

Author:  erik945 [ Thu Jul 06, 2017 5:36 pm ]
Post subject:  Re: Deus EX: Mankind Divided .archive

Thank you for your tool
Do you planed update texture convertor?


And what does it mean when in a namemap in a line with a numeric file name there are several symbolic names?
as example:

71795574635566022 00FF11B1DA9913C6 [assembly:/_g2_assets/templates/aspectdummy.aspect]([assembly:/environment_assets/20_props/10_cluttering/17_industrial/bucket_laundry_garbage_a/bucket_laundry_garbage_a.template?/bucket_laundry_garbage_a bf7f7ac8-1407-421c-bb9a-faa85ce5fcdb.entitytemplate].entitytype,[modules:/runtime.streaming.moduleinfo?zstreamingroot.class].entitytype,[modules:/runtime.activation.moduleinfo?zactivationroot.class].entitytype).pc_entitytype

6031250991359965 00156D640CE427DD [[assembly:/material_class/2dnsr_pb.uberclass?/blendwithvertexcolor#!diffusemap1##tint,df~0,u~wrap,v~wrap,f~21,uv~uv0!!diffusemap2#bo~maskblend,bc~r,#tint,col!!usenormalmapfiltering#!nmf~1,normalmap1##tint,df~2,u~wrap,v~wrap,f~21,uv~uv0!!normalmap2#bo~add,#tint,df~3,u~wrap,v~wrap,f~21,uv~uv0!!specular1##tint,col!roughness#tint,df~5,u~wrap,v~wrap,f~21,uv~uv0!!specular2#bc~r,#tint,col!roughness#tint,df~7,u~wrap,v~wrap,f~21,uv~uv0,ch~g!!masktypesettings#!masktexture##tint,col!!maskchannelr#channelmap#tint,uf~7!!alphatypesettings#!blendwithvertexalpha#!vertexshadertype#!materialtype1#mt~dielectric!dbm~opaque.ubervariant].fx](dx11).pc_mate

Author:  volfin [ Thu Jul 06, 2017 6:14 pm ]
Post subject:  Re: Deus EX: Mankind Divided .archive

as far as I know, there's no need to update the texture tool. it works with all valid textures. If that's not true, I need more information.

As for the number stuff, I have no idea what all that is.

Author:  erik945 [ Thu Jul 06, 2017 7:59 pm ]
Post subject:  Re: Deus EX: Mankind Divided .archive

Convertor don't work with textures extracted by Sir Kanes tool.

Author:  volfin [ Thu Jul 06, 2017 8:07 pm ]
Post subject:  Re: Deus EX: Mankind Divided .archive

erik945 wrote:
Convertor don't work with textures extracted by Sir Kanes tool.


yes everything was made for the original quickbms script. for that it all works.

Author:  erik945 [ Sun Jul 09, 2017 3:11 pm ]
Post subject:  Re: Deus EX: Mankind Divided .archive

It seems to me Sir Kane's extractor misses some files.
I can not find some models - for example, weapons and augmentations from the screen of equipment.

Author:  Sir Kane [ Tue Jul 18, 2017 12:14 am ]
Post subject:  Re: Deus EX: Mankind Divided .archive

http://sktest.aruarose.com/DXMDExtract_v002.7z

  • Fixed a bug that caused files in containers whose names contain upper case characters to not be extracted.
  • Speculative fix for files containing wrong data.
  • Now generates a list of external resources and their file names.
  • Now generates a binary file containing the data required to map resource IDs to asset names.

This is the format for the ResMap.bin file:
Code:
   uint32_t   Count;
   struct {
      uint64_t   ResourceID;
      uint32_t   NameLength;
      char      Name[NameLength];
   } Entries[Count];

The name is zero terminated.

Author:  Vindis [ Wed Jul 19, 2017 1:15 pm ]
Post subject:  Re: Deus EX: Mankind Divided .archive

I made a texture tool for Sir Kanes extractor (fortunately, the data is already there, uncompressed :D ).
It have basic functionality, and convert dxt1,5, bc4,5,7 textures. All you have to do is rename texture bin files to ".tex"
Just put the files into the same folder as the exe, and it converts all tex to DDS. (you can figure out what is texture from names list)


PS: Can't found files for these types, so it's not converting these tho, and you need intel's photoshop plugin to work with BCx dds.

Code:
AttemptedFormat = TEXTURE::U8bpp;
      else if (VAPHeader->Format == 0x1D)
         AttemptedFormat = TEXTURE::U32bpp;

      if (AttemptedFormat == TEXTURE::NO_FORMAT)
      {
         printf("Unknown Format %d\n", VAPHeader->Format);
         continue;
      }

Author:  erik945 [ Wed Jul 19, 2017 7:13 pm ]
Post subject:  Re: Deus EX: Mankind Divided .archive

Join the topic - a couple of scripts on the python 2.

The first - copies and renames the extracted files. The result is a correct directory tree.
Fdir = "E: \\ DXMD_DLC_UNPACK" # base unpack folder with nameMap file
Fsrc = "E: \\ md_files_DLC2 \\" # folder with correct filenames


The second for the blender - batch export of 3d models to fbx
Folder to search (recursively)
Fdir = "E: \\ md_files_DLC \\ objects \\" - directory for search


:D

Author:  Vindis [ Sun Jul 23, 2017 6:49 pm ]
Post subject:  Re: Deus EX: Mankind Divided .archive

@Volfin

Hi Volfin,

may I ask you a big favor?
If you have time, could you please write down the file format of the model files?
I've tried to figure out, and write a template in 010 Editor, but it drives me nuts. Sometimes it works,
sometimes the values are offset by an unknown value (the tables for example, it points to an address, there's no vertex number).

I don't know nothing about python scripting, so your importer is not a good starting point for me :(

I would really appreciate your help!

Author:  volfin [ Sun Jul 23, 2017 8:17 pm ]
Post subject:  Re: Deus EX: Mankind Divided .archive

Vindis wrote:
@Volfin

Hi Volfin,

may I ask you a big favor?
If you have time, could you please write down the file format of the model files?
I've tried to figure out, and write a template in 010 Editor, but it drives me nuts. Sometimes it works,
sometimes the values are offset by an unknown value (the tables for example, it points to an address, there's no vertex number).

I don't know nothing about python scripting, so your importer is not a good starting point for me :(

I would really appreciate your help!


This is because there is no one format. There's 10 to 15 distinct vertex buffer formats. And they can have various sub blocks per format. so it's a very messy thing to decode. I have no real documentation, I figure a block format out in 010, then add it to the python script. then when I or someone else encounters another format, I add it to the script as well. The only real documentation is the python script. It's not hard to read really.

I have a library of data readers in the python script:

Code:
def readcstr(): #Null Terminated text string
def BB(n): #unsigned Char
def Bb(n): # Signed Char
def BH(n): # Unsigned Short
def Bh(n): # Signed Short
def Bf(n): # Float
def BF(n): # HALF FLoat
def Bi(n): # Signed Long Int
def BI(n): # Unsigned Long Int
def Bd(n): # Double
def alignPosition(alignment): # align to given byte width (4/8/16 etc)


if you see BB(3) in the code, I read in 3 bytes, but didn't assign them to a variable, so I just skipped them
if you see new_var = BB(3), then 'new_var' is now an array of 3 bytes read in. new_var[0] would be the first new_var[1] would be the second etc.
if you see new_var = BB(1)[0], then I immediately assigned the value read in to 'new_var' instead of assigning it as an array.

filehandle.seek(next_offset,0) # SEEK_SET
filehandle.seek(hdr_size,1) #SEEK_CUR
filehandle.seek(0,2) # SEEK_END
These all do exactly what they do in C, seek on the file stream either from the start (SET), the current position (CUR), or from the end (END).

The rest should be self explanatory.

Author:  Vindis [ Mon Jul 24, 2017 9:30 am ]
Post subject:  Re: Deus EX: Mankind Divided .archive

volfin wrote:
Vindis wrote:
@Volfin

Hi Volfin,

may I ask you a big favor?
If you have time, could you please write down the file format of the model files?
I've tried to figure out, and write a template in 010 Editor, but it drives me nuts. Sometimes it works,
sometimes the values are offset by an unknown value (the tables for example, it points to an address, there's no vertex number).

I don't know nothing about python scripting, so your importer is not a good starting point for me :(

I would really appreciate your help!


This is because there is no one format. There's 10 to 15 distinct vertex buffer formats. And they can have various sub blocks per format. so it's a very messy thing to decode. I have no real documentation, I figure a block format out in 010, then add it to the python script. then when I or someone else encounters another format, I add it to the script as well. The only real documentation is the python script. It's not hard to read really.

I have a library of data readers in the python script:

Code:
def readcstr(): #Null Terminated text string
def BB(n): #unsigned Char
def Bb(n): # Signed Char
def BH(n): # Unsigned Short
def Bh(n): # Signed Short
def Bf(n): # Float
def BF(n): # HALF FLoat
def Bi(n): # Signed Long Int
def BI(n): # Unsigned Long Int
def Bd(n): # Double
def alignPosition(alignment): # align to given byte width (4/8/16 etc)


if you see BB(3) in the code, I read in 3 bytes, but didn't assign them to a variable, so I just skipped them
if you see new_var = BB(3), then 'new_var' is now an array of 3 bytes read in. new_var[0] would be the first new_var[1] would be the second etc.
if you see new_var = BB(1)[0], then I immediately assigned the value read in to 'new_var' instead of assigning it as an array.

filehandle.seek(next_offset,0) # SEEK_SET
filehandle.seek(hdr_size,1) #SEEK_CUR
filehandle.seek(0,2) # SEEK_END
These all do exactly what they do in C, seek on the file stream either from the start (SET), the current position (CUR), or from the end (END).

The rest should be self explanatory.


Oooh!

The binary reading part avoided my attention. Now it's really not that hard :)

Thank You very much!

Page 6 of 10 All times are UTC + 1 hour
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
http://www.phpbb.com/