XeNTaX Forum Index
Forum MultiEx Commander Tools Tools Home
It is currently Tue Oct 23, 2018 1:02 pm

All times are UTC + 1 hour


Forum rules


Please click here to view the forum rules



Post new topic Reply to topic  [ 43 posts ]  Go to page 1, 2, 3  Next
Author Message
 Post subject: JX3 HD Remake 剑网3 (.dat files)
PostPosted: Fri Sep 28, 2018 12:26 am 
Offline
beginner
User avatar

Joined: Sat Mar 25, 2017 1:40 am
Posts: 23
Has thanked: 6 times
Have thanks: 1 time
EDITED (10/3/2018):
Just put this out there so if anyone can write a Noesis/Max plugin to import/export this game then here's some mode samples 3MB (https://drive.google.com/open?id=1hrUJM ... tPD7NvzE9M)...
Here's the full folder with everything extracted 743MB ==> https://drive.google.com/open?id=16XYbB ... TMxPmfxd7G
----------------------------------------------------------------------------------------------------------
Hi, I know this game has multiple versions in the past... but this JX3 HD Remake look really awesome and I'm just posted here cuz I checked the other posts and I don't think it's been cover yet (or sorry if I missed it).

This game has a total of 74.5GB of .DAT files and .IDX files (mostly .dat files... I don't know if these dat files can just be imported into Noesis/Max/Blender by writing some script on it or maybe it need to be extracted from the dat files first). So if anyone can help extract these DAT files, then you have my thanks and anyone who is interested in the future.

Sample file 202MB https://drive.google.com/open?id=1PD7W4hxY1HF5c0EoOCPBhgL-15LbQkWH

If need more sample, just let me know... I'll upload them ASAP.
Here's some Screenshots
ImageImageImageImage
http://image.9game.cn/2018/3/2/19669711.jpg
http://jx3.xoyo.com/uploadfile/2018/030 ... 524187.jpg
http://jx3.xoyo.com/uploadfile/2018/030 ... 519705.jpg
-----------------------------------------------------------
Game Site: http://jx3.xoyo.com/index/
MMOByte Overview: https://www.youtube.com/watch?v=MIv731kIFu4

---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
All the necessary tool(s) from this thread is here. https://drive.google.com/open?id=18UGXT ... cWytrWQFuP


Last edited by petventh18 on Fri Oct 05, 2018 1:40 am, edited 4 times in total.

Top
 Profile  
 
 Post subject: Re: JX3 HD Remake 剑网3 (.dat files)
PostPosted: Fri Sep 28, 2018 1:39 pm 
Offline
ultra-veteran
ultra-veteran
User avatar

Joined: Tue Jul 05, 2016 9:37 am
Posts: 369
Has thanked: 16 times
Have thanks: 206 times
You can use this BMS script to unpack the chunks as a first step:

Just select the dat file as input and leave the rest to the script.

No time to dig into the format. Sorry about that.


Attachments:


You do not have the required permissions to view the files attached to this post. Register to gain access.


_________________
"No investigation means no right to speak."
- An Imitable Workflow for Reverse Engineering a Game Model
- Reverse Model Wireframe


Say it with action: click the Image when you get helped.


Top
 Profile  
 
 Post subject: Re: JX3 HD Remake 剑网3 (.dat files)
PostPosted: Fri Sep 28, 2018 4:55 pm 
Offline
double-veteran
double-veteran
User avatar

Joined: Wed Jan 05, 2011 12:41 pm
Posts: 665
Has thanked: 39 times
Have thanks: 693 times
Looks pretty cool, could someone share model samples after extraction?

_________________
Making model-import scripts, PM


Top
 Profile  
 
 Post subject: Re: JX3 HD Remake 剑网3 (.dat files)
PostPosted: Fri Sep 28, 2018 5:03 pm 
Offline
M-M-M-Monster veteran
M-M-M-Monster veteran
User avatar

Joined: Fri Apr 20, 2012 9:24 am
Posts: 2446
Location: Nexus, searching for Jim Kirk
Has thanked: 587 times
Have thanks: 1260 times
I didn't find face indices in the orgUncompressed folder at a first glance.
Here's some file types in the orgCompressed folder:


Attachments:


You do not have the required permissions to view the files attached to this post. Register to gain access.


_________________
Bigchillghost, Reverse Engineering a Game Model: http://forum.xentax.com/viewtopic.php?f=29&t=17889
extracting simple models: http://forum.xentax.com/viewtopic.php?f=29&t=10894
Make_H2O-ForzaHor3-jm9.zip


Top
 Profile  
 
 Post subject: Re: JX3 HD Remake 剑网3 (.dat files)
PostPosted: Fri Sep 28, 2018 5:24 pm 
Offline
ultra-veteran
ultra-veteran
User avatar

Joined: Tue Jul 05, 2016 9:37 am
Posts: 369
Has thanked: 16 times
Have thanks: 206 times
The raw data size of chunk 0 is around 110 mb while it's almost the equal size after decompression. Maybe there's another layer of compression. Chunks with smaller size seem OK though, like in chunk_1 there're some texture name strings and material info.
Edit: Never mind. It's just a mp4 file.

BTW, data of some chunks in the orgUncompressed folder seem to be in big-endian.

_________________
"No investigation means no right to speak."
- An Imitable Workflow for Reverse Engineering a Game Model
- Reverse Model Wireframe


Say it with action: click the Image when you get helped.


Last edited by Bigchillghost on Tue Oct 02, 2018 11:18 am, edited 1 time in total.

Top
 Profile  
 
 Post subject: Re: JX3 HD Remake 剑网3 (.dat files)
PostPosted: Mon Oct 01, 2018 7:05 pm 
Offline
beginner
User avatar

Joined: Sat Mar 25, 2017 1:40 am
Posts: 23
Has thanked: 6 times
Have thanks: 1 time
Hi thanks a lot guys... I've used the datUnpack.bms script and got a lot of chunks .dat files from it. There's like 250+ folders with their respective .dat files from the game. So it's gonna take a while to find where the models are.
I'll upload some samples to here and see if anyone can make sense of anything.
https://drive.google.com/open?id=1647lR ... iTcAQOMVPn
It would be easier to identify the models through the textures but since there's like 300 files to extract from, I'll upload it once I got any lead.
In the mean time, here's some additional samples (not extracted)
https://drive.google.com/open?id=14Oafz ... YdgQNsexPk
https://drive.google.com/open?id=1kNlFI ... KN8Q_6nmTF
https://drive.google.com/open?id=1pNsT2 ... LZDFWJNHiB
https://drive.google.com/open?id=1YbOpx ... zXw7tcZZPa


Top
 Profile  
 
 Post subject: Re: JX3 HD Remake 剑网3 (.dat files)
PostPosted: Mon Oct 01, 2018 9:23 pm 
Offline
beginner
User avatar

Joined: Sat Mar 25, 2017 1:40 am
Posts: 23
Has thanked: 6 times
Have thanks: 1 time
Bigchillghost wrote:
You can use this BMS script to unpack the chunks as a first step:

Just select the dat file as input and leave the rest to the script.

No time to dig into the format. Sorry about that.


Hi Bigchillghost, thanks for the script...
I kept running into these issues running out of memory so I can't extract all these .dat files... I had 32GB RAM on my machine so I kinda doubt that's the main issue.
Sample https://drive.google.com/open?id=1cV4Xz ... c0prvQ3Cgb
Image

And sometime it show the message
"Not enough storage is available to process this command." but I don't remember which file it was so I can't provide you a sample.

I've used both the normal quickbms.exe and quickbms_4gb.exe... They pretty much the same... So I guess some of the header got messed up in these files or something.


Top
 Profile  
 
 Post subject: Re: JX3 HD Remake 剑网3 (.dat files)
PostPosted: Tue Oct 02, 2018 12:00 pm 
Offline
ultra-veteran
ultra-veteran
User avatar

Joined: Tue Jul 05, 2016 9:37 am
Posts: 369
Has thanked: 16 times
Have thanks: 206 times
petventh18 wrote:
In the mean time, here's some additional samples (not extracted)

The script works fine with these files(tested on the first sample). You just need to wait patiently before you witness any output info.
petventh18 wrote:
Hi Bigchillghost, thanks for the script...
I kept running into these issues running out of memory so I can't extract all these .dat files... I had 32GB RAM on my machine so I kinda doubt that's the main issue.

I modified the script to export correct file extensions this morning, but when I was testing it, that issue occurs to me on the same file that my script handled correctly twice before. I guess it might be some bugs of QuickBMS. So I decided to make a standalone tool for unpacking the assets. It's roughly tested on several samples(the one in the first post and the one from the first url) but it's still a beta.

_________________
"No investigation means no right to speak."
- An Imitable Workflow for Reverse Engineering a Game Model
- Reverse Model Wireframe


Say it with action: click the Image when you get helped.


Top
 Profile  
 
 Post subject: Re: JX3 HD Remake 剑网3 (.dat files)
PostPosted: Tue Oct 02, 2018 12:14 pm 
Offline
ultra-veteran
ultra-veteran
User avatar

Joined: Tue Jul 05, 2016 9:37 am
Posts: 369
Has thanked: 16 times
Have thanks: 206 times
Here's a x86 release of the tool:

Place them in the dir of the dat/idx files and double-click the .bat to run the program in batch command.
It'll log all text info into a new corresponding file(.txt or .json) within 8 megabytes for convenience of reading, unless current text chunk has already exceed that size.

Note: Files end with ".TBD" means their extensions are to be determined.


Attachments:


You do not have the required permissions to view the files attached to this post. Register to gain access.


_________________
"No investigation means no right to speak."
- An Imitable Workflow for Reverse Engineering a Game Model
- Reverse Model Wireframe


Say it with action: click the Image when you get helped.


Last edited by Bigchillghost on Wed Oct 03, 2018 5:16 am, edited 1 time in total.

Top
 Profile  
 
 Post subject: Re: JX3 HD Remake 剑网3 (.dat files)
PostPosted: Tue Oct 02, 2018 7:53 pm 
Offline
beginner
User avatar

Joined: Sat Mar 25, 2017 1:40 am
Posts: 23
Has thanked: 6 times
Have thanks: 1 time
Bigchillghost wrote:
Here's a x86 release of the tool:
Place them in the dir of the dat/idx files and double-click the .bat to run the program in batch command.
It'll log all text info into one corresponding file(.txt or .json) for convenience of reading.

Oh thanks so much, work like a charm... I tested and worked on most files... Here's the files that had a model textures and some .mesh files along with some subtitle scripts too...
60MB ===> https://drive.google.com/open?id=1FcQyb ... LvWG2SNMFH
743MB ==> https://drive.google.com/open?id=16XYbB ... TMxPmfxd7G
Image
...with this maybe someone can help write a Noesis/Max script to import/export them.

There's still some files that came out empty such as this screenshot... but since it can already extracted the textures and mesh files from some other .dat files, I guess these files not necessarily contain any more models (I hope)... I guess the only thing I might miss is if these are animation files or something.
Image
Here's a sample of the file that can't be extracted.
596MB ==> https://drive.google.com/open?id=12i5GJ ... 8n2Pn3IWIe


Top
 Profile  
 
 Post subject: Re: JX3 HD Remake 剑网3 (.dat files)
PostPosted: Wed Oct 03, 2018 5:17 am 
Offline
ultra-veteran
ultra-veteran
User avatar

Joined: Tue Jul 05, 2016 9:37 am
Posts: 369
Has thanked: 16 times
Have thanks: 206 times
petventh18 wrote:
There's still some files that came out empty such as this screenshot... but since it can already extracted the textures and mesh files from some other .dat files, I guess these files not necessarily contain any more models (I hope)... I guess the only thing I might miss is if these are animation files or something.
Here's a sample of the file that can't be extracted.

Well it's just an issue resulting from the illegal extensions. I've improved the code for determining them, especially more accurately for the mesh files so you might need to run the lastest version on all packages again. Sorry about that.

_________________
"No investigation means no right to speak."
- An Imitable Workflow for Reverse Engineering a Game Model
- Reverse Model Wireframe


Say it with action: click the Image when you get helped.


Top
 Profile  
 
 Post subject: Re: JX3 HD Remake 剑网3 (.dat files)
PostPosted: Wed Oct 03, 2018 10:31 am 
Offline
ultra-veteran
ultra-veteran
User avatar

Joined: Tue Jul 05, 2016 9:37 am
Posts: 369
Has thanked: 16 times
Have thanks: 206 times
petventh18 wrote:
Here's the files that had a model textures and some .mesh files along with some subtitle scripts too...
743MB ==> https://drive.google.com/open?id=16XYbB ... TMxPmfxd7G

Mesh from 000003B3.mesh:

And the one from 000005E0.mesh:
Image
Image

It seems that the model needs proper scaling to get the perfect mesh.


Attachments:


You do not have the required permissions to view the files attached to this post. Register to gain access.


_________________
"No investigation means no right to speak."
- An Imitable Workflow for Reverse Engineering a Game Model
- Reverse Model Wireframe


Say it with action: click the Image when you get helped.


Last edited by Bigchillghost on Fri Oct 05, 2018 6:33 am, edited 1 time in total.

Top
 Profile  
 
 Post subject: Re: JX3 HD Remake 剑网3 (.dat files)
PostPosted: Wed Oct 03, 2018 11:22 am 
Offline
M-M-M-Monster veteran
M-M-M-Monster veteran
User avatar

Joined: Fri Apr 20, 2012 9:24 am
Posts: 2446
Location: Nexus, searching for Jim Kirk
Has thanked: 587 times
Have thanks: 1260 times
Bigchillghost wrote:
It seems that the model needs proper scaling to get the perfect mesh.
I don't even know how this can work: floats for the mesh don't fit into an FVF of 6?

_________________
Bigchillghost, Reverse Engineering a Game Model: http://forum.xentax.com/viewtopic.php?f=29&t=17889
extracting simple models: http://forum.xentax.com/viewtopic.php?f=29&t=10894
Make_H2O-ForzaHor3-jm9.zip


Top
 Profile  
 
 Post subject: Re: JX3 HD Remake 剑网3 (.dat files)
PostPosted: Wed Oct 03, 2018 11:36 am 
Offline
ultra-veteran
ultra-veteran
User avatar

Joined: Tue Jul 05, 2016 9:37 am
Posts: 369
Has thanked: 16 times
Have thanks: 206 times
shakotay2 wrote:
I don't even know how this can work: floats for the mesh don't fit into an FVF of 6?

You mean the 6 floats before the vertex buffer?

_________________
"No investigation means no right to speak."
- An Imitable Workflow for Reverse Engineering a Game Model
- Reverse Model Wireframe


Say it with action: click the Image when you get helped.


Top
 Profile  
 
 Post subject: Re: JX3 HD Remake 剑网3 (.dat files)
PostPosted: Wed Oct 03, 2018 12:11 pm 
Offline
M-M-M-Monster veteran
M-M-M-Monster veteran
User avatar

Joined: Fri Apr 20, 2012 9:24 am
Posts: 2446
Location: Nexus, searching for Jim Kirk
Has thanked: 587 times
Have thanks: 1260 times
FVFsize= 6; but I see, it's ShortAll (instead of WordUV, which you seem to have used for uvs' displaying only.)

My confusion also aroused from the fact that there seems to be 6 floats values (bounding box) before the mesh using shorts.

_________________
Bigchillghost, Reverse Engineering a Game Model: http://forum.xentax.com/viewtopic.php?f=29&t=17889
extracting simple models: http://forum.xentax.com/viewtopic.php?f=29&t=10894
Make_H2O-ForzaHor3-jm9.zip


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 43 posts ]  Go to page 1, 2, 3  Next

All times are UTC + 1 hour


Who is online

Users browsing this forum: coderxxx, zory16 and 5 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group