READ THE RULES: Click here

Follow us on Facebook: https://www.facebook.com/xentax/ :)

Final Fantasy XI .DAT (PC)

Post questions about game models here, or help out others!
paulgswanson
ultra-n00b
Posts: 2
Joined: Sun Sep 04, 2011 9:00 pm
Has thanked: 1 time
Been thanked: 2 times

Re: Final Fantasy XI .DAT (PC)

Post by paulgswanson » Wed Apr 09, 2014 3:11 pm

Thanks for the link. there was quite a bit of helpful info in there, Google translate makes my head hurt though. I'm assuming those scripts only worked on character models with skeletons? The translation chrome gave me was....difficult.

TehDave
advanced
Posts: 73
Joined: Tue Apr 21, 2009 5:24 am
Has thanked: 10 times
Been thanked: 5 times

Re: Final Fantasy XI .DAT (PC)

Post by TehDave » Thu Jul 03, 2014 11:56 am

paulgswanson wrote:Thanks for the link. there was quite a bit of helpful info in there, Google translate makes my head hurt though. I'm assuming those scripts only worked on character models with skeletons? The translation chrome gave me was....difficult.
Lucky you, I can't even get the page to load.

Any chance you have the scripts and would be willing to upload them?

Humm08
ultra-n00b
Posts: 3
Joined: Sun Aug 24, 2014 8:23 am
Been thanked: 1 time

Re: Final Fantasy XI .DAT (PC)

Post by Humm08 » Fri Oct 17, 2014 7:10 pm

Was the source for Altana View ever posted anywhere?

alloy
ultra-n00b
Posts: 1
Joined: Thu Aug 14, 2014 9:30 am
Been thanked: 1 time

Re: Final Fantasy XI .DAT (PC)

Post by alloy » Tue Oct 21, 2014 4:24 am

Back like in 2007 ffxitool had its source code up somewhere on the japanese net.
Maybe it still is if you know japanese google-fu

http://cosmosa.jp/ffxitool/

Just a bunch on tools that can play music load models, levels and all kinds of data for ffxi.

The guy who wrote Altana had access to the ffxitool source. His prog is just a lot more polished.

But this is were it all ffxi data ripping started.

Also the ffxi tool page i posted up there has some links to what seems to be info on data structure and whatnot.

And you can find a link to dl the actual programs on the page i linked up there

http://cosmosa.jp/ffxitool/kaimemo.html
http://cosmosa.jp/ffxitool/generator.html
http://cosmosa.jp/ffxitool/note.html

Hope anyone here can make any sense or if its even useful to you guys.

And also theres a page dedicated to what i guess is his tool version for ff14

http://cosmosa.jp/ffxivtool/

I have no idea what it does because i don't have that game and i literally just found out about it lol.

User avatar
MrAdults
Moderator
Posts: 1007
Joined: Mon Mar 23, 2009 2:57 am
Has thanked: 44 times
Been thanked: 498 times

Re: Final Fantasy XI .DAT (PC)

Post by MrAdults » Sat Oct 10, 2015 9:15 pm

I ended up re-reversing this damn thing. It'll be in the Noesis release I'm planning to put up in a few minutes.

Image Image
Image Image

Animation and maps and everything are supported. Still haven't totally figured out how blending is specified for certain map objects, so some stuff may or may not be blended when it shouldn't or should be. But I did figure out terrain blending, so that's nice. I came across a code snippet for MZB/MMB "decryption" after extensive Japanese Googling, which I'm pretty sure was written by the author of ffxitools, but am not sure. But, thanks to that guy for being able to load the encrypted maps, I guess.

Character sets are also supported, via a new format called "ff11datset". An example sample_hume01.ff11datset will be included in the Noesis scenes directory. It works like this:

Code: Select all

NOESIS_FF11_DAT_SET
;^ must be the first line of the file

;search for dats using a path retrieved from a registry key
setPathKey	"HKEY_LOCAL_MACHINE" "SOFTWARE\PlayOnlineUS\InstallFolder" "0001"

;search for dats on a path relative to this file 
;setPathRel	"./"

;search for dats on an absolute path
;setPathAbs	"c:/whatever/ff11/"

dat			"__skeleton"	"ROM/27/82.dat"
dat			"__animation"	"ROM/27/82.dat"
dat			"face"			"ROM/27/87.dat"
dat			"head"			"ROM/27/103.dat"
dat			"body"			"ROM/28/7.dat"
dat			"hands"			"ROM/28/52.dat"
dat			"waist"			"ROM/28/84.dat"
dat			"legs"			"ROM/28/116.dat"
dat			"weapon"		"ROM/29/20.dat"
I'll probably put together a little Python script at some point that just tosses up a GUI to let you scroll through body parts and just generates+loads these files as you scroll through.

User avatar
MrAdults
Moderator
Posts: 1007
Joined: Mon Mar 23, 2009 2:57 am
Has thanked: 44 times
Been thanked: 498 times

Re: Final Fantasy XI .DAT (PC)

Post by MrAdults » Sat Oct 10, 2015 11:40 pm

Here's the source:

http://www.richwhitehouse.com/index.php ... t=Snippets

Try not to pretend you wrote it. Unbelievable how many people I came across showing viewers they "wrote" and refusing to send them to people, when clearly all they did was find an old source distribution of ffxitools. Probably some interesting behavioral statistics to be collected there.

falconcool
veteran
Posts: 87
Joined: Tue Jun 16, 2009 4:41 am
Has thanked: 11 times
Been thanked: 6 times

Re: Final Fantasy XI .DAT (PC)

Post by falconcool » Wed Oct 14, 2015 12:49 am

Here is my maxscript to load hi-poly character model in charcter creation.
It still doen't support skeleton /animation.
I'm not good at matrix calculation.I've discussed it with fatduck in his dead forum before,still not finished yet.
MrAdult,hope you can add them into noesis too.Tyvm for doing this project.

Code: Select all

--if (heapSize < 200000) then
--heapSize = 2000000 -- allow ~ 40 MB instead of just 7.5 MB. Prevents"Runtime Error: Out of scripter memory"
fname = getOpenFileName \ 
caption:"FFXI Model File" \
types:" FFXI Character Model File(*.dat)|*.dat|All files (*.*)|*.*|" \

clearlistener()
f = fopen fname "rb"


------------------------------------------Function Define Start
-------------------------------------------------------------------------------------
fn ReadFixedString bstream fixedLen =
(
   local str = ""
   for i = 1 to fixedLen do
   (
      str += bit.intAsChar (ReadByte bstream #unsigned)
   )
   str
)

-------------------------------------------------------------------------------------
------------------------------------------Function Define End
global Meshpointer_array = #()
global Bonepointer_array = #()
global Skinpointer_array  = #()
f_size = getFileSize fname
print "File size"
print f_size
f_lines = (f_size / 16)  as integer
fseek f 0 #seek_set
for i = 1 to (f_lines-1) do (
	pointer_pos = (ftell f) as integer
	magic_string1 = readlong f #unsigned
	fseek f 6 #seek_cur
	magic_num = readshort f #unsigned
	fseek f 4 #seek_cur
	if  magic_string1 == 1162629459 do( ------check if "SQLE"
		
		if magic_num == 10 do( -----check if mesh
			append Meshpointer_array [pointer_pos,0]
		)	----if magic_num = 10
		
		if magic_num == 11 do( --------check if skeleton
			append Bonepointer_array [pointer_pos,0]
		)----	if magic_num == 11
		
		if magic_num == 21 do( ------check if skin
			append Skinpointer_array [pointer_pos,0]
		) -----	if magic_num == 21				
	)----if  magic_string1 == 1162629459
)----or i = 1 to f_lines 
fseek f 0 #seek_set


for i = 1 to (Meshpointer_array.count) do (
  global Vert_array = #()
  global Normal_array = #()
  global UV_array = #()
  global Face_array=#()  
  global facechunk_array =#()
    
  mesh_start = Meshpointer_array [i][1] as integer
	
  fseek f mesh_start #seek_set------------------offset to chunk starting position

  fseek f 96 #seek_cur----------------------------where start to read mesh

  vtcount = readlong f #unsigned
  fseek f 12 #seek_cur
  print "vertice count check"
  print vtcount

    for j=1 to vtcount do( ----read vert
        vx = readfloat f
        vy = readfloat f
        vz = readfloat f
       append Vert_array [vx,vy,vz]	   
	)	----for j=1 to vtcount
	
   fseek f 8 #seek_cur	

	
	 for j=1 to vtcount do( ----read normal
        nx = readfloat f
        ny = readfloat f
        nz = readfloat f
	   append Normal_array[nx,ny,nz]
	 )--for j=1 to vtcount
	 
  fseek f 8 #seek_cur	 
    for j = 1 to vtcount do ( ----readUV
	     Uvx= ( ( readfloat f ) * -1 ) + 1
	     Uvy= readfloat f
	     --Uvy -= 1
	     append UV_array[Uvx,Uvy,0]
	)----j = 1 to vtcount

   code_count = readlong f #unsigned--------------------------------------face index start
   section_count = readlong f #unsigned
	
    for j = 1 to section_count do(
	    fid_pos = ftell f
	    fid_count = readshort f #signed
			
	     if fid_count < 0 do  (   ---------------------------- tri-strip
	      fid_num = fid_count * -1
	      fseek f 2 #seek_cur----skip 3 bytes of 0xff
		  backface=1 
		  global fid_array =#()
			 
			 for k = 1 to fid_num do (
				 fid = ( readlong f #unsigned )+ 1
				 append fid_array[fid,0]				 
			 )----for k = 1 to fid_num
			 
			 for k = 1 to (fid_num - 2) do(
				 f1 = fid_array[k][1]
				 f2 = fid_array[k+1][1]
				 f3 = fid_array[k+2][1]
				 backface=backface*-1
				   				   
                   if f1!=f2 and f1!=f3 and f2!=f3 then(
					   
                        if (backface==1) then (   
                           append Face_array [f1,f2,f3]
                        )    
    
                        if (backface==-1) then (
                           append Face_array [f1,f3,f2]
                        )						
                    )	----if f1!=f2 and f1!=f3 and f2!=f3
			 )----	 for k = 1 to (fid_num - 2)
	     )----if fid_count < 0 
		
	if fid_count > 0 do ( ---------------------------------------------trilist array
	  fseek f fid_pos #seek_set
	  print "Trilist array found ! position check!"
	  print fid_pos
		
	  fid_count = readshort f #signed
	    if fid_count < 0 do(
			
	      fid_num = fid_count * -1	
	    )----if fid_count < 0
	  
       fseek f 2 #seek_cur----skip 2 bytes of 0xff	
	  fid_num = fid_count
		print fid_num
	   for k = 1 to (fid_num / 3) do (
		   f1 = (readlong f #unsigned ) + 1
		   f2 = (readlong f #unsigned )+ 1
		   f3 = (readlong f #unsigned )+ 1
		   append Face_array [f1,f2,f3]
		   		   
	   )---- for k = 1 to (fid_count / 3)
	   
	   ) ----if fid_count>0 
    )----	 for j = 1 to section_count

  -------------------------------------------------------------------------------------build mesh
  -----------------------------------------------------------------------------------
  msh = mesh vertices:Vert_array faces:Face_array
  msh.numTVerts = UV_array.count
  buildTVFaces msh
    for j = 1 to UV_array.count     do setTVert  msh j UV_array[j]
    for j = 1 to Face_array.count   do setTVFace msh j Face_array[j]
    for j = 1 to Normal_array.count do setNormal msh j Normal_array[j]
  --msh.name = chunk_name
  
)----for i = 1 to (Meshpointer_array.count)




gc()
fflush f
fclose f

And here is the data list you can find those hi-poly models/animations.
You do not have the required permissions to view the files attached to this post.

User avatar
MrAdults
Moderator
Posts: 1007
Joined: Mon Mar 23, 2009 2:57 am
Has thanked: 44 times
Been thanked: 498 times

Re: Final Fantasy XI .DAT (PC)

Post by MrAdults » Wed Oct 14, 2015 1:59 am

Huh? Noesis already supports all of the character parts, and piecing them together as mentioned above. And animations. And levels.

falconcool
veteran
Posts: 87
Joined: Tue Jun 16, 2009 4:41 am
Has thanked: 11 times
Been thanked: 6 times

Re: Final Fantasy XI .DAT (PC)

Post by falconcool » Wed Oct 14, 2015 2:12 am

MrAdults wrote:Huh? Noesis already supports all of the character parts, and piecing them together as mentioned above. And animations. And levels.
The character creation has high poly models different with normal in game models.Check those files in the data list i added please:)

User avatar
MrAdults
Moderator
Posts: 1007
Joined: Mon Mar 23, 2009 2:57 am
Has thanked: 44 times
Been thanked: 498 times

Re: Final Fantasy XI .DAT (PC)

Post by MrAdults » Wed Oct 14, 2015 2:20 am

Seems your list is wrong for the US/NA release (or maybe just wrong), then. For example 63/104 isn't mesh data. You should refer to them by file ID's from the tables instead of the .dat names to be consistent.

falconcool
veteran
Posts: 87
Joined: Tue Jun 16, 2009 4:41 am
Has thanked: 11 times
Been thanked: 6 times

Re: Final Fantasy XI .DAT (PC)

Post by falconcool » Wed Oct 14, 2015 3:13 am

MrAdults wrote:Seems your list is wrong for the US/NA release (or maybe just wrong), then. For example 63/104 isn't mesh data. You should refer to them by file ID's from the tables instead of the .dat names to be consistent.
ya,,some of the data in the list is wrong,,but their number is close
63/104 is texture
try 63/103 with my script
Image

User avatar
MrAdults
Moderator
Posts: 1007
Joined: Mon Mar 23, 2009 2:57 am
Has thanked: 44 times
Been thanked: 498 times

Re: Final Fantasy XI .DAT (PC)

Post by MrAdults » Wed Oct 14, 2015 3:45 am

I figured that other animation format out, but it's pretty awful and worse than the in-game one. Not sure why it exists.

I might add support to Noesis at some point, but I don't see much point, cause it's self-evident and has nothing in common with the main .dats. Maybe I'll just do it in an external script.

falconcool
veteran
Posts: 87
Joined: Tue Jun 16, 2009 4:41 am
Has thanked: 11 times
Been thanked: 6 times

Re: Final Fantasy XI .DAT (PC)

Post by falconcool » Sat Oct 17, 2015 5:56 am

MrAdults wrote:I figured that other animation format out, but it's pretty awful and worse than the in-game one. Not sure why it exists.

I might add support to Noesis at some point, but I don't see much point, cause it's self-evident and has nothing in common with the main .dats. Maybe I'll just do it in an external script.

I'm guessing that's was done by 2 different teams.1st team design the high poly models,2nd team use those models to design in game low-poly count models with optimized animation format.Because this game was designed to be working on very old ps2 with very limited ram size 10 years ago.
I viewed some maps from rom1 folder and found some blending issue you mentioned.Some trees leaves still lacking of correct alpha channel.And lacking of water materials.
Here is the actual in game video of those map area.Hope it will help your coding,Regards:)
https://www.youtube.com/watch?v=kzrmwcSizIY

User avatar
MrAdults
Moderator
Posts: 1007
Joined: Mon Mar 23, 2009 2:57 am
Has thanked: 44 times
Been thanked: 498 times

Re: Final Fantasy XI .DAT (PC)

Post by MrAdults » Sat Oct 17, 2015 6:33 am

I meant, the format itself is worse. It's bigger and offers worse compression quality for pbchannel or limited functionality for framechannel. Considering the other format had to have existed prior to ship, there's no reason for this format to exist in tandem, even if the content pipeline was entirely different.

TehDave
advanced
Posts: 73
Joined: Tue Apr 21, 2009 5:24 am
Has thanked: 10 times
Been thanked: 5 times

Re: Final Fantasy XI .DAT (PC)

Post by TehDave » Wed Oct 21, 2015 8:55 am

Oh shit, did it finally happen? Can I finally get the models out with weights and animations?

Always wanted to do stuff with FFXI's models but my lack of rigging expertise limited me.

Post Reply