JX3 HD Remake 剑网3 (.dat files)

[petventh18]

EDITED (10/3/2018):
Just put this out there so if anyone can write a Noesis/Max plugin to import/export this game then here's some mode samples 3MB (https://drive.google.com/open?id=1hrUJM ... tPD7NvzE9M)...
Here's the full folder with everything extracted 743MB ==> https://drive.google.com/open?id=16XYbB ... TMxPmfxd7G
----------------------------------------------------------------------------------------------------------
Hi, I know this game has multiple versions in the past... but this JX3 HD Remake look really awesome and I'm just posted here cuz I checked the other posts and I don't think it's been cover yet (or sorry if I missed it).

This game has a total of 74.5GB of .DAT files and .IDX files (mostly .dat files... I don't know if these dat files can just be imported into Noesis/Max/Blender by writing some script on it or maybe it need to be extracted from the dat files first). So if anyone can help extract these DAT files, then you have my thanks and anyone who is interested in the future.

Sample file 202MB https://drive.google.com/open?id=1PD7W4 ... L-15LbQkWH

If need more sample, just let me know... I'll upload them ASAP.
Here's some Screenshots

http://image.9game.cn/2018/3/2/19669711.jpg
http://jx3.xoyo.com/uploadfile/2018/030 ... 524187.jpg
http://jx3.xoyo.com/uploadfile/2018/030 ... 519705.jpg
-----------------------------------------------------------
Game Site: http://jx3.xoyo.com/index/
MMOByte Overview: https://www.youtube.com/watch?v=MIv731kIFu4

---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
All the necessary tool(s) from this thread is here. https://drive.google.com/open?id=18UGXT ... cWytrWQFuP

[Bigchillghost]

You can use this BMS script to unpack the chunks as a first step:

datUnpack.rar

Just select the dat file as input and leave the rest to the script.

No time to dig into the format. Sorry about that.

[zaramot]

Looks pretty cool, could someone share model samples after extraction?

[shakotay2]

I didn't find face indices in the orgUncompressed folder at a first glance.
Here's some file types in the orgCompressed folder:

JX3-data.jpg

[Bigchillghost]

The raw data size of chunk 0 is around 110 mb while it's almost the equal size after decompression. Maybe there's another layer of compression. Chunks with smaller size seem OK though, like in chunk_1 there're some texture name strings and material info.
Edit: Never mind. It's just a mp4 file.

BTW, data of some chunks in the orgUncompressed folder seem to be in big-endian.

[petventh18]

Hi thanks a lot guys... I've used the datUnpack.bms script and got a lot of chunks .dat files from it. There's like 250+ folders with their respective .dat files from the game. So it's gonna take a while to find where the models are.
I'll upload some samples to here and see if anyone can make sense of anything.
https://drive.google.com/open?id=1647lR ... iTcAQOMVPn
It would be easier to identify the models through the textures but since there's like 300 files to extract from, I'll upload it once I got any lead.
In the mean time, here's some additional samples (not extracted)
https://drive.google.com/open?id=14Oafz ... YdgQNsexPk
https://drive.google.com/open?id=1kNlFI ... KN8Q_6nmTF
https://drive.google.com/open?id=1pNsT2 ... LZDFWJNHiB
https://drive.google.com/open?id=1YbOpx ... zXw7tcZZPa

[petventh18]

You can use this BMS script to unpack the chunks as a first step:

datUnpack.rar

Just select the dat file as input and leave the rest to the script.

No time to dig into the format. Sorry about that.

Hi Bigchillghost, thanks for the script...
I kept running into these issues running out of memory so I can't extract all these .dat files... I had 32GB RAM on my machine so I kinda doubt that's the main issue.
Sample https://drive.google.com/open?id=1cV4Xz ... c0prvQ3Cgb


And sometime it show the message
"Not enough storage is available to process this command." but I don't remember which file it was so I can't provide you a sample.

I've used both the normal quickbms.exe and quickbms_4gb.exe... They pretty much the same... So I guess some of the header got messed up in these files or something.

[Bigchillghost]

In the mean time, here's some additional samples (not extracted)

The script works fine with these files(tested on the first sample). You just need to wait patiently before you witness any output info.

Hi Bigchillghost, thanks for the script...
I kept running into these issues running out of memory so I can't extract all these .dat files... I had 32GB RAM on my machine so I kinda doubt that's the main issue.

I modified the script to export correct file extensions this morning, but when I was testing it, that issue occurs to me on the same file that my script handled correctly twice before. I guess it might be some bugs of QuickBMS. So I decided to make a standalone tool for unpacking the assets. It's roughly tested on several samples(the one in the first post and the one from the first url) but it's still a beta.

[Bigchillghost]

Here's a x86 release of the tool:

JX3HDUnpacker.7z

Place them in the dir of the dat/idx files and double-click the .bat to run the program in batch command.
It'll log all text info into a new corresponding file(.txt or .json) within 8 megabytes for convenience of reading, unless current text chunk has already exceed that size.

Note: Files end with ".TBD" means their extensions are to be determined.

[petventh18]

Here's a x86 release of the tool:
Place them in the dir of the dat/idx files and double-click the .bat to run the program in batch command.
It'll log all text info into one corresponding file(.txt or .json) for convenience of reading.

Oh thanks so much, work like a charm... I tested and worked on most files... Here's the files that had a model textures and some .mesh files along with some subtitle scripts too...
60MB ===> https://drive.google.com/open?id=1FcQyb ... LvWG2SNMFH
743MB ==> https://drive.google.com/open?id=16XYbB ... TMxPmfxd7G

...with this maybe someone can help write a Noesis/Max script to import/export them.

There's still some files that came out empty such as this screenshot... but since it can already extracted the textures and mesh files from some other .dat files, I guess these files not necessarily contain any more models (I hope)... I guess the only thing I might miss is if these are animation files or something.

Here's a sample of the file that can't be extracted.
596MB ==> https://drive.google.com/open?id=12i5GJ ... 8n2Pn3IWIe

[Bigchillghost]

There's still some files that came out empty such as this screenshot... but since it can already extracted the textures and mesh files from some other .dat files, I guess these files not necessarily contain any more models (I hope)... I guess the only thing I might miss is if these are animation files or something.
Here's a sample of the file that can't be extracted.

Well it's just an issue resulting from the illegal extensions. I've improved the code for determining them, especially more accurately for the mesh files so you might need to run the lastest version on all packages again. Sorry about that.

[Bigchillghost]

Here's the files that had a model textures and some .mesh files along with some subtitle scripts too...
743MB ==> https://drive.google.com/open?id=16XYbB ... TMxPmfxd7G

Mesh from 000003B3.mesh:

000003B3.mesh.jpg

And the one from 000005E0.mesh:



It seems that the model needs proper scaling to get the perfect mesh.

[shakotay2]

It seems that the model needs proper scaling to get the perfect mesh.

I don't even know how this can work: floats for the mesh don't fit into an FVF of 6?

[Bigchillghost]

I don't even know how this can work: floats for the mesh don't fit into an FVF of 6?

You mean the 6 floats before the vertex buffer?

[shakotay2]

FVFsize= 6; but I see, it's ShortAll (instead of WordUV, which you seem to have used for uvs' displaying only.)

My confusion also aroused from the fact that there seems to be 6 floats values (bounding box) before the mesh using shorts.