XeNTaX Forum Index
Forum MultiEx Commander Tools Tools Home
It is currently Fri Nov 17, 2017 8:40 pm

All times are UTC + 1 hour


Forum rules


Please click here to view the forum rules



Post new topic Reply to topic  [ 118 posts ]  Go to page Previous  1 ... 3, 4, 5, 6, 7, 8  Next
Author Message
 Post subject: Re: Deus EX: Mankind Divided .archive
PostPosted: Sat May 06, 2017 1:48 pm 
Offline
advanced

Joined: Fri Apr 08, 2011 9:31 pm
Posts: 43
Has thanked: 15 times
Have thanks: 46 times














You can make the ads go away by registering

volfin wrote:
your command line is fine. But it would seem it's not finding any valid *.vap files in the 'input' folder. And just in case you think it will parse down subdirectories inside the 'input' folder, be assured it will not. It will only process *.vap files in 'input' folder itself. If your *.vap files are in input\someother directory, then you need to specify that directly.


Hi Volfin,

Tried to convert all txet (assuming those are the texture files) but I got an error, "unknown format -1" for each one. I renamed every file to vap.
I extracted the files with Sir Kanes' extractor, could be that problem? Should I try quickbms?

Thank you for your help!


Top
 Profile  
 
 Post subject: Re: Deus EX: Mankind Divided .archive
PostPosted: Sat May 06, 2017 5:06 pm 
Offline
ultra-veteran
ultra-veteran
User avatar

Joined: Sun Jul 06, 2014 6:30 am
Posts: 344
Has thanked: 73 times
Have thanks: 220 times
Vindis wrote:
volfin wrote:
your command line is fine. But it would seem it's not finding any valid *.vap files in the 'input' folder. And just in case you think it will parse down subdirectories inside the 'input' folder, be assured it will not. It will only process *.vap files in 'input' folder itself. If your *.vap files are in input\someother directory, then you need to specify that directly.


Hi Volfin,

Tried to convert all txet (assuming those are the texture files) but I got an error, "unknown format -1" for each one. I renamed every file to vap.
I extracted the files with Sir Kanes' extractor, could be that problem? Should I try quickbms?

Thank you for your help!


yes everything was written for the quickbms files. I don't know the differences involved with the other extractor.


Top
 Profile  
 
 Post subject: Re: Deus EX: Mankind Divided .archive
PostPosted: Wed May 17, 2017 4:23 pm 
Offline
advanced

Joined: Fri Apr 08, 2011 9:31 pm
Posts: 43
Has thanked: 15 times
Have thanks: 46 times
volfin wrote:
Vindis wrote:
volfin wrote:
your command line is fine. But it would seem it's not finding any valid *.vap files in the 'input' folder. And just in case you think it will parse down subdirectories inside the 'input' folder, be assured it will not. It will only process *.vap files in 'input' folder itself. If your *.vap files are in input\someother directory, then you need to specify that directly.


Hi Volfin,

Tried to convert all txet (assuming those are the texture files) but I got an error, "unknown format -1" for each one. I renamed every file to vap.
I extracted the files with Sir Kanes' extractor, could be that problem? Should I try quickbms?

Thank you for your help!


yes everything was written for the quickbms files. I don't know the differences involved with the other extractor.


Is there any chance, that you share the file structure of the texture files, so I can write a program to work with Sir Kanes' extractor? Maybe there's not much difference...


Top
 Profile  
 
 Post subject: Re: Deus EX: Mankind Divided .archive
PostPosted: Wed May 17, 2017 5:13 pm 
Offline
ultra-veteran
ultra-veteran
User avatar

Joined: Sun Jul 06, 2014 6:30 am
Posts: 344
Has thanked: 73 times
Have thanks: 220 times
Vindis wrote:
Is there any chance, that you share the file structure of the texture files, so I can write a program to work with Sir Kanes' extractor? Maybe there's not much difference...


The header is as follows:

Code:
struct HeaderVAP
{
   long Unknown0;
   short Unknown1;
   short Unknown2;
   short Unknown7;
   short Unknown8;
   short Width;
   short Height;
   short Format1; // MIP count maybe?
   short Format;
   long Unknown3;
   long Unknown4;
   long Unknown5;
   long Unknown6;
};


Directly after the header comes the compressed texture data. How that data is formatted is dependent on the "Format" variable. The known "Format" codes are:

Code:
      TEXTURE AttemptedFormat = TEXTURE::NO_FORMAT;
      if (VAPHeader->Format == 0x4A)
         AttemptedFormat = TEXTURE::DXT1;
      else if (VAPHeader->Format == 0x50)
         AttemptedFormat = TEXTURE::DXT5;
      else if (VAPHeader->Format == 0x53)
         AttemptedFormat = TEXTURE::BC4;
      else if (VAPHeader->Format == 0x56)
         AttemptedFormat = TEXTURE::BC5;
      else if (VAPHeader->Format == 0x5B)
         AttemptedFormat = TEXTURE::BC7;
      else if (VAPHeader->Format == 0x43)
         AttemptedFormat = TEXTURE::U8bpp;
      else if (VAPHeader->Format == 0x1D)
         AttemptedFormat = TEXTURE::U32bpp;

      if (AttemptedFormat == TEXTURE::NO_FORMAT)
      {
         printf("Unknown Format %d\n", VAPHeader->Format);
         continue;
      }


Top
 Profile  
 
 Post subject: Re: Deus EX: Mankind Divided .archive
PostPosted: Thu Jul 06, 2017 5:36 pm 
Offline
veteran

Joined: Fri Jan 20, 2012 5:43 pm
Posts: 130
Has thanked: 19 times
Have thanks: 36 times
Thank you for your tool
Do you planed update texture convertor?


And what does it mean when in a namemap in a line with a numeric file name there are several symbolic names?
as example:

71795574635566022 00FF11B1DA9913C6 [assembly:/_g2_assets/templates/aspectdummy.aspect]([assembly:/environment_assets/20_props/10_cluttering/17_industrial/bucket_laundry_garbage_a/bucket_laundry_garbage_a.template?/bucket_laundry_garbage_a bf7f7ac8-1407-421c-bb9a-faa85ce5fcdb.entitytemplate].entitytype,[modules:/runtime.streaming.moduleinfo?zstreamingroot.class].entitytype,[modules:/runtime.activation.moduleinfo?zactivationroot.class].entitytype).pc_entitytype

6031250991359965 00156D640CE427DD [[assembly:/material_class/2dnsr_pb.uberclass?/blendwithvertexcolor#!diffusemap1##tint,df~0,u~wrap,v~wrap,f~21,uv~uv0!!diffusemap2#bo~maskblend,bc~r,#tint,col!!usenormalmapfiltering#!nmf~1,normalmap1##tint,df~2,u~wrap,v~wrap,f~21,uv~uv0!!normalmap2#bo~add,#tint,df~3,u~wrap,v~wrap,f~21,uv~uv0!!specular1##tint,col!roughness#tint,df~5,u~wrap,v~wrap,f~21,uv~uv0!!specular2#bc~r,#tint,col!roughness#tint,df~7,u~wrap,v~wrap,f~21,uv~uv0,ch~g!!masktypesettings#!masktexture##tint,col!!maskchannelr#channelmap#tint,uf~7!!alphatypesettings#!blendwithvertexalpha#!vertexshadertype#!materialtype1#mt~dielectric!dbm~opaque.ubervariant].fx](dx11).pc_mate


Top
 Profile  
 
 Post subject: Re: Deus EX: Mankind Divided .archive
PostPosted: Thu Jul 06, 2017 6:14 pm 
Offline
ultra-veteran
ultra-veteran
User avatar

Joined: Sun Jul 06, 2014 6:30 am
Posts: 344
Has thanked: 73 times
Have thanks: 220 times
as far as I know, there's no need to update the texture tool. it works with all valid textures. If that's not true, I need more information.

As for the number stuff, I have no idea what all that is.


Top
 Profile  
 
 Post subject: Re: Deus EX: Mankind Divided .archive
PostPosted: Thu Jul 06, 2017 7:59 pm 
Offline
veteran

Joined: Fri Jan 20, 2012 5:43 pm
Posts: 130
Has thanked: 19 times
Have thanks: 36 times
Convertor don't work with textures extracted by Sir Kanes tool.


Top
 Profile  
 
 Post subject: Re: Deus EX: Mankind Divided .archive
PostPosted: Thu Jul 06, 2017 8:07 pm 
Offline
ultra-veteran
ultra-veteran
User avatar

Joined: Sun Jul 06, 2014 6:30 am
Posts: 344
Has thanked: 73 times
Have thanks: 220 times
erik945 wrote:
Convertor don't work with textures extracted by Sir Kanes tool.


yes everything was made for the original quickbms script. for that it all works.


Top
 Profile  
 
 Post subject: Re: Deus EX: Mankind Divided .archive
PostPosted: Sun Jul 09, 2017 3:11 pm 
Offline
veteran

Joined: Fri Jan 20, 2012 5:43 pm
Posts: 130
Has thanked: 19 times
Have thanks: 36 times
It seems to me Sir Kane's extractor misses some files.
I can not find some models - for example, weapons and augmentations from the screen of equipment.


Top
 Profile  
 
 Post subject: Re: Deus EX: Mankind Divided .archive
PostPosted: Tue Jul 18, 2017 12:14 am 
Offline
veteran

Joined: Mon Aug 06, 2012 4:14 am
Posts: 95
Has thanked: 0 time
Have thanks: 62 times
http://sktest.aruarose.com/DXMDExtract_v002.7z

  • Fixed a bug that caused files in containers whose names contain upper case characters to not be extracted.
  • Speculative fix for files containing wrong data.
  • Now generates a list of external resources and their file names.
  • Now generates a binary file containing the data required to map resource IDs to asset names.

This is the format for the ResMap.bin file:
Code:
   uint32_t   Count;
   struct {
      uint64_t   ResourceID;
      uint32_t   NameLength;
      char      Name[NameLength];
   } Entries[Count];

The name is zero terminated.


Top
 Profile  
 
 Post subject: Re: Deus EX: Mankind Divided .archive
PostPosted: Wed Jul 19, 2017 1:15 pm 
Offline
advanced

Joined: Fri Apr 08, 2011 9:31 pm
Posts: 43
Has thanked: 15 times
Have thanks: 46 times
I made a texture tool for Sir Kanes extractor (fortunately, the data is already there, uncompressed :D ).
It have basic functionality, and convert dxt1,5, bc4,5,7 textures. All you have to do is rename texture bin files to ".tex"
Just put the files into the same folder as the exe, and it converts all tex to DDS. (you can figure out what is texture from names list)


PS: Can't found files for these types, so it's not converting these tho, and you need intel's photoshop plugin to work with BCx dds.

Code:
AttemptedFormat = TEXTURE::U8bpp;
      else if (VAPHeader->Format == 0x1D)
         AttemptedFormat = TEXTURE::U32bpp;

      if (AttemptedFormat == TEXTURE::NO_FORMAT)
      {
         printf("Unknown Format %d\n", VAPHeader->Format);
         continue;
      }


Attachments:


You do not have the required permissions to view the files attached to this post. Register to gain access.



Top
 Profile  
 
 Post subject: Re: Deus EX: Mankind Divided .archive
PostPosted: Wed Jul 19, 2017 7:13 pm 
Offline
veteran

Joined: Fri Jan 20, 2012 5:43 pm
Posts: 130
Has thanked: 19 times
Have thanks: 36 times
Join the topic - a couple of scripts on the python 2.

The first - copies and renames the extracted files. The result is a correct directory tree.
Fdir = "E: \\ DXMD_DLC_UNPACK" # base unpack folder with nameMap file
Fsrc = "E: \\ md_files_DLC2 \\" # folder with correct filenames


The second for the blender - batch export of 3d models to fbx
Folder to search (recursively)
Fdir = "E: \\ md_files_DLC \\ objects \\" - directory for search


:D


Attachments:


You do not have the required permissions to view the files attached to this post. Register to gain access.



Top
 Profile  
 
 Post subject: Re: Deus EX: Mankind Divided .archive
PostPosted: Sun Jul 23, 2017 6:49 pm 
Offline
advanced

Joined: Fri Apr 08, 2011 9:31 pm
Posts: 43
Has thanked: 15 times
Have thanks: 46 times
@Volfin

Hi Volfin,

may I ask you a big favor?
If you have time, could you please write down the file format of the model files?
I've tried to figure out, and write a template in 010 Editor, but it drives me nuts. Sometimes it works,
sometimes the values are offset by an unknown value (the tables for example, it points to an address, there's no vertex number).

I don't know nothing about python scripting, so your importer is not a good starting point for me :(

I would really appreciate your help!


Top
 Profile  
 
 Post subject: Re: Deus EX: Mankind Divided .archive
PostPosted: Sun Jul 23, 2017 8:17 pm 
Offline
ultra-veteran
ultra-veteran
User avatar

Joined: Sun Jul 06, 2014 6:30 am
Posts: 344
Has thanked: 73 times
Have thanks: 220 times
Vindis wrote:
@Volfin

Hi Volfin,

may I ask you a big favor?
If you have time, could you please write down the file format of the model files?
I've tried to figure out, and write a template in 010 Editor, but it drives me nuts. Sometimes it works,
sometimes the values are offset by an unknown value (the tables for example, it points to an address, there's no vertex number).

I don't know nothing about python scripting, so your importer is not a good starting point for me :(

I would really appreciate your help!


This is because there is no one format. There's 10 to 15 distinct vertex buffer formats. And they can have various sub blocks per format. so it's a very messy thing to decode. I have no real documentation, I figure a block format out in 010, then add it to the python script. then when I or someone else encounters another format, I add it to the script as well. The only real documentation is the python script. It's not hard to read really.

I have a library of data readers in the python script:

Code:
def readcstr(): #Null Terminated text string
def BB(n): #unsigned Char
def Bb(n): # Signed Char
def BH(n): # Unsigned Short
def Bh(n): # Signed Short
def Bf(n): # Float
def BF(n): # HALF FLoat
def Bi(n): # Signed Long Int
def BI(n): # Unsigned Long Int
def Bd(n): # Double
def alignPosition(alignment): # align to given byte width (4/8/16 etc)


if you see BB(3) in the code, I read in 3 bytes, but didn't assign them to a variable, so I just skipped them
if you see new_var = BB(3), then 'new_var' is now an array of 3 bytes read in. new_var[0] would be the first new_var[1] would be the second etc.
if you see new_var = BB(1)[0], then I immediately assigned the value read in to 'new_var' instead of assigning it as an array.

filehandle.seek(next_offset,0) # SEEK_SET
filehandle.seek(hdr_size,1) #SEEK_CUR
filehandle.seek(0,2) # SEEK_END
These all do exactly what they do in C, seek on the file stream either from the start (SET), the current position (CUR), or from the end (END).

The rest should be self explanatory.


Top
 Profile  
 
 Post subject: Re: Deus EX: Mankind Divided .archive
PostPosted: Mon Jul 24, 2017 9:30 am 
Offline
advanced

Joined: Fri Apr 08, 2011 9:31 pm
Posts: 43
Has thanked: 15 times
Have thanks: 46 times
volfin wrote:
Vindis wrote:
@Volfin

Hi Volfin,

may I ask you a big favor?
If you have time, could you please write down the file format of the model files?
I've tried to figure out, and write a template in 010 Editor, but it drives me nuts. Sometimes it works,
sometimes the values are offset by an unknown value (the tables for example, it points to an address, there's no vertex number).

I don't know nothing about python scripting, so your importer is not a good starting point for me :(

I would really appreciate your help!


This is because there is no one format. There's 10 to 15 distinct vertex buffer formats. And they can have various sub blocks per format. so it's a very messy thing to decode. I have no real documentation, I figure a block format out in 010, then add it to the python script. then when I or someone else encounters another format, I add it to the script as well. The only real documentation is the python script. It's not hard to read really.

I have a library of data readers in the python script:

Code:
def readcstr(): #Null Terminated text string
def BB(n): #unsigned Char
def Bb(n): # Signed Char
def BH(n): # Unsigned Short
def Bh(n): # Signed Short
def Bf(n): # Float
def BF(n): # HALF FLoat
def Bi(n): # Signed Long Int
def BI(n): # Unsigned Long Int
def Bd(n): # Double
def alignPosition(alignment): # align to given byte width (4/8/16 etc)


if you see BB(3) in the code, I read in 3 bytes, but didn't assign them to a variable, so I just skipped them
if you see new_var = BB(3), then 'new_var' is now an array of 3 bytes read in. new_var[0] would be the first new_var[1] would be the second etc.
if you see new_var = BB(1)[0], then I immediately assigned the value read in to 'new_var' instead of assigning it as an array.

filehandle.seek(next_offset,0) # SEEK_SET
filehandle.seek(hdr_size,1) #SEEK_CUR
filehandle.seek(0,2) # SEEK_END
These all do exactly what they do in C, seek on the file stream either from the start (SET), the current position (CUR), or from the end (END).

The rest should be self explanatory.


Oooh!

The binary reading part avoided my attention. Now it's really not that hard :)

Thank You very much!


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 118 posts ]  Go to page Previous  1 ... 3, 4, 5, 6, 7, 8  Next

All times are UTC + 1 hour


Who is online

Users browsing this forum: No registered users and 5 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group