READ THE RULES: Click here

Follow us on Facebook: https://www.facebook.com/xentax/ :)

Extracting simple models

Read or post any tutorial related to file format analysis for modding purposes.
User avatar
Tosyk
double-veteran
double-veteran
Posts: 934
Joined: Thu Oct 22, 2009 10:24 am
Location: Russia, Siberia
Has thanked: 225 times
Been thanked: 109 times
Contact:

Re: Extracting simple models

Post by Tosyk » Tue Jul 31, 2018 7:51 pm

I don't understand why finding faces is in the first step?

for me the easiest process is something like this:

1. step
- put vertex block start
- put vertex count nubmer
- choose some options depends on how strider work/look (float/half-float)

-------------at this point I can see vertex cloud---------------

2. step
- put face block start
- put face count number
- choose some options depends on how face strider work/look (byte word/dword)

---------------at this point I can see full mesh-----------------

3. step
- put uv info
- put vertex count number from 1 step
- choose some options depends on how vertex strider work/look

----------at this point I can see full mesh with uv-------------


h2o is a great tool but its ui looks overwhelmed and inconsistent — it had steps though a lot of options is just floating around. and some ui elements are hidden and is not clear what buttons related where. even 2 simple lines separated 3 steps could make ui clearer.
Thank you for all you do here
my blog | my forum

User avatar
shakotay2
MEGAVETERAN
MEGAVETERAN
Posts: 2730
Joined: Fri Apr 20, 2012 9:24 am
Location: Nexus, searching for Jim Kirk
Has thanked: 675 times
Been thanked: 1407 times

Re: Extracting simple models

Post by shakotay2 » Tue Jul 31, 2018 8:22 pm

First of all many thanks for your reply! I was just reformatting/editing my post, not seeing that you answered already, sorry for that.
Tosyk wrote:I don't understand why finding faces is in the first step?
well, because it's the simplest part.
Finding the scrambled alphabet is easy, you can't deny that seriously.
(Simply review forums post; first successful action of most newbies dealing with hex2obj is finding the face indices.)
-------------at this point I can see vertex cloud---------------
Agreed. Look here: viewtopic.php?f=16&t=18031&p=140097&hil ... ud#p140097

It's not forbidden to start with "point clouding". (I should mention it in the tutorial, that's correct, thanks!)

3. step
- put uv info
- put vertex count number from 1 step
- choose some options depends on how vertex strider work/look

----------at this point I can see full mesh with uv-------------
I see, you like it structured. :) In the first years of hex2obj I didn't care too much for uvs.
Presenting the mesh was the main goal.
So taking care for the uvs at the end of the process (although "placed" at step 2) has "historical reasons".
Also you don't need to put vertex count (for uvs) in hex2obj because it uses it automatically (which is ok for simple models).
it had steps though a lot of options is just floating around. and some ui elements are hidden and is not clear what buttons related where. even 2 simple lines separated 3 steps could make ui clearer.
I've mentioned those "floating" options in my edit. (yeah, this was a matter of laziness)

I'll see what I can change concerning the ui arrangement, but don't expect it too soon.
Also I don't think that I'll change the general approach because it made sense for me for hundreds of (simple) models now.

Thank you for your post, really appreciated!
Bigchillghost, Reverse Engineering a Game Model: viewtopic.php?f=29&t=17889
extracting simple models: viewtopic.php?f=29&t=10894
Make_H2O-ForzaHor3-jm9.zip
"You quoted the whole thing, what a mess."

User avatar
Tosyk
double-veteran
double-veteran
Posts: 934
Joined: Thu Oct 22, 2009 10:24 am
Location: Russia, Siberia
Has thanked: 225 times
Been thanked: 109 times
Contact:

Re: Extracting simple models

Post by Tosyk » Tue Jul 31, 2018 9:25 pm

shakotay2 wrote:First of all many thanks for your reply! I was just reformatting/editing my post, not seeing that you answered already, sorry for that.
Tosyk wrote:I don't understand why finding faces is in the first step?
well, because it's the simplest part.
Finding the scrambled alphabet is easy, you can't deny that seriously.
(Simply review forums post; first successful action of most newbies dealing with hex2obj is finding the face indices.)
I do agree — I found that "alphabet" myself many years ago. Unfortunately knowing that info will lead nowhere within h2o, to be honest and that's why "step 1" felt really confusing from the very beginning
shakotay2 wrote:
-------------at this point I can see vertex cloud---------------
Agreed. Look here: viewtopic.php?f=16&t=18031&p=140097&hil ... ud#p140097

It's not forbidden to start with "point clouding". (I should mention it in the tutorial, that's correct, thanks!)
yeah but it's "step 3"! and start from "step 3" is a lot more confusing than lack of feedback from "step 1" and "step 2"
shakotay2 wrote:
3. step
- put uv info
- put vertex count number from 1 step
- choose some options depends on how vertex strider work/look

----------at this point I can see full mesh with uv-------------
I see, you like it structured. :) In the first years of hex2obj I didn't care too much for uvs.
Presenting the mesh was the main goal.
So taking care for the uvs at the end of the process (although "placed" at step 2) has "historical reasons".
Also you don't need to put vertex count (for uvs) in hex2obj because it uses it automatically (which is ok for simple models).
if I don't need to put vertex count for uv the ui should show me this in inactive field in which I will see same vertex count as I found for vertex buffer. See, that's why I always started from h2o tutorial :/
shakotay2 wrote:
it had steps though a lot of options is just floating around. and some ui elements are hidden and is not clear what buttons related where. even 2 simple lines separated 3 steps could make ui clearer.
I've mentioned those "floating" options in my edit. (yeah, this was a matter of laziness)

I'll see what I can change concerning the ui arrangement, but don't expect it too soon.
Also I don't think that I'll change the general approach because it made sense for me for hundreds of (simple) models now.

Thank you for your post, really appreciated!
I understand that h2o's steps it's conventionality and I can fill data in any order but for any ui - especially advanced one - the visual feedback is very important - I would say the most important. If ui confusing user from the very first step the user will drop it but if the ui with every its step will give user some feedack - a result - it will give a better understanding of ui and reduce amount of questions on how to use it.

Anyways, all you do here is absolutely briliant. I'm following the forum and know all the local gurus :mrgreen:

p.s.: sorry for too sirious threat, I understand this thing is just for fun, but wanted to leave here my thoughts :)
Thank you for all you do here
my blog | my forum

User avatar
shakotay2
MEGAVETERAN
MEGAVETERAN
Posts: 2730
Joined: Fri Apr 20, 2012 9:24 am
Location: Nexus, searching for Jim Kirk
Has thanked: 675 times
Been thanked: 1407 times

Re: Extracting simple models

Post by shakotay2 » Tue Jul 31, 2018 10:53 pm

Tosyk wrote:I do agree — I found that "alphabet" myself many years ago. Unfortunately knowing that info will lead nowhere within h2o
well, once again, this steps leads you to the maximum face index which is generally identical with the vertex count. The vertex count (vc) together with the size of the vertex data block (sizeOfVD) leads you to the FVF size.
(where FVFsize= sizeOfVD/vc, but not always, that's one reason why newbies fail.)
yeah but it's "step 3"! and start from "step 3" is a lot more confusing than lack of feedback from "step 1" and "step 2"
well, maybe I should talk of "recommended" order of steps but I see, trying to convince people that hex2obj is a helper tool, if they don't feel so doesn't make too much sense.
Anyways, all you do here is absolutely briliant.
From your posts I don't feel so :wink: but anyways I think you're a dedicated follower of the Model Researcher, it's ok. You should use the tool, that gives you the best results.
Bigchillghost, Reverse Engineering a Game Model: viewtopic.php?f=29&t=17889
extracting simple models: viewtopic.php?f=29&t=10894
Make_H2O-ForzaHor3-jm9.zip
"You quoted the whole thing, what a mess."

User avatar
Tosyk
double-veteran
double-veteran
Posts: 934
Joined: Thu Oct 22, 2009 10:24 am
Location: Russia, Siberia
Has thanked: 225 times
Been thanked: 109 times
Contact:

Re: Extracting simple models

Post by Tosyk » Wed Aug 01, 2018 9:39 am

shakotay2 wrote:
Anyways, all you do here is absolutely briliant.
From your posts I don't feel so :wink:
I'm always excited with people knowledge and trying to input my best. But I see you helping people in every thread and that is really nice
shakotay2 wrote:but anyways I think you're a dedicated follower of the Model Researcher, it's ok. You should use the tool, that gives you the best results.
actually maxscript is the tool I understand better than any others :D
Thank you for all you do here
my blog | my forum

User avatar
Tosyk
double-veteran
double-veteran
Posts: 934
Joined: Thu Oct 22, 2009 10:24 am
Location: Russia, Siberia
Has thanked: 225 times
Been thanked: 109 times
Contact:

Re: Extracting simple models

Post by Tosyk » Fri Aug 03, 2018 8:36 am

sorry for dp

I start to make a maxscript for MMB format of Division since python script is outdated: viewtopic.php?f=16&t=14374
Then I realized that at this point of my knowledge I'm able to read (and even debug) some python code and for the last few hours before sleep I interpreted full MMB header from python to the maxscript. though I can't understand some python functions:

first of all, what 'bs.read("9I")' mean? what is "9I" or "10I"?

Code: Select all

for j in range(meshNumGroups):
   if fileType == 9 or fileType == 10:
        meshGroupInfo.append(bs.read("9I"))
   elif fileType == 11:
        meshGroupInfo.append(bs.read("10I"))
second of all, where the script starts read vertex stride?

here's what I have so far:
division_v00_header.zip
You do not have the required permissions to view the files attached to this post.
Thank you for all you do here
my blog | my forum

User avatar
Acewell
VIP member
VIP member
Posts: 1263
Joined: Wed Nov 05, 2008 12:16 pm
Has thanked: 2392 times
Been thanked: 730 times

Re: Extracting simple models

Post by Acewell » Fri Aug 03, 2018 12:09 pm

Tosyk wrote:what 'bs.read("9I")' mean?
it means read nine unsigned integers for a total of 36 bytes (9 * 4) :D
if you do print(meshGroupInfo) after reading it should list the 9 integers in log.

User avatar
Tosyk
double-veteran
double-veteran
Posts: 934
Joined: Thu Oct 22, 2009 10:24 am
Location: Russia, Siberia
Has thanked: 225 times
Been thanked: 109 times
Contact:

Re: Extracting simple models

Post by Tosyk » Fri Aug 03, 2018 12:14 pm

AceWell wrote:
Tosyk wrote:what 'bs.read("9I")' mean?
it means read nine unsigned integers for a total of 36 bytes (9 * 4) :D
if you do print(meshGroupInfo) after reading it should list the 9 integers in log.
dank a bunch :mrgreen: so it can be done by bs.seek(36, NOESEEK_REL) as well, correct?

now I only need to undestand a loop by which noesis read vertex/face buffers to build a mesh
Thank you for all you do here
my blog | my forum

User avatar
chrrox
Moderator
Posts: 2541
Joined: Sun May 18, 2008 3:01 pm
Has thanked: 57 times
Been thanked: 1269 times

Re: Extracting simple models

Post by chrrox » Fri Aug 03, 2018 9:48 pm

Tosyk wrote:
AceWell wrote:
Tosyk wrote:what 'bs.read("9I")' mean?
it means read nine unsigned integers for a total of 36 bytes (9 * 4) :D
if you do print(meshGroupInfo) after reading it should list the 9 integers in log.
dank a bunch :mrgreen: so it can be done by bs.seek(36, NOESEEK_REL) as well, correct?

now I only need to undestand a loop by which noesis read vertex/face buffers to build a mesh

bs.seek(36, NOESEEK_REL) would skip those bytes and not read them.

__NPExample.txt shows 2 ways to read meshes.

style 1 like max scripts
meshes = []
for i in range(0, numMeshes):
for j in range(0, numIdx): - faces / triangles
idxList.append(bs.readInt())
for j in range(0, numPos): - vert position
posList.append(NoeVec3.fromBytes(bs.readBytes(12)))
mesh = NoeMesh(idxList, posList, meshName, meshMat)
for j in range(0, numNrm): - vert normals
mesh.normals.append(NoeVec3.fromBytes(bs.readBytes(12)))
for j in range(0, numUVs): - vert uv's
mesh.uvs.append(NoeVec3.fromBytes(bs.readBytes(12)))
meshes.append(mesh)

or the easier way
for i in range(0, numMeshes):
triangles = bs.readBytes(numIdx * 4)
positions = bs.readBytes(numPos * 12)
normals = bs.readBytes(numPos * 12) if numNrm == numPos else None
uvs = bs.readBytes(numPos * 12) if numUVs == numPos else None
rapi.rpgBindPositionBuffer(positions, noesis.RPGEODATA_FLOAT, 12)
rapi.rpgBindNormalBuffer(normals, noesis.RPGEODATA_FLOAT, 12)
rapi.rpgBindUV1Buffer(uvs, noesis.RPGEODATA_FLOAT, 12)
rapi.rpgCommitTriangles(triangles, noesis.RPGEODATA_INT, numIdx, noesis.RPGEO_TRIANGLE, 1)
rapi.rpgClearBufferBinds() #reset in case a subsequent mesh doesn't have the same components
mdl = rapi.rpgConstructModel()

User avatar
Tosyk
double-veteran
double-veteran
Posts: 934
Joined: Thu Oct 22, 2009 10:24 am
Location: Russia, Siberia
Has thanked: 225 times
Been thanked: 109 times
Contact:

Re: Extracting simple models

Post by Tosyk » Mon Aug 06, 2018 9:32 am

chrrox, thanks for clarify this a bit

so what this line (meshGroupInfo.append(bs.read("9I"))) exactly do? can I read vertex buffer (12 bytes in stride) the same way? like (bs.read("3I"))?

it's unfortune that noesis doesn't have documentation for its python functions nor detailed/explained examples
Thank you for all you do here
my blog | my forum

User avatar
chrrox
Moderator
Posts: 2541
Joined: Sun May 18, 2008 3:01 pm
Has thanked: 57 times
Been thanked: 1269 times

Re: Extracting simple models

Post by chrrox » Mon Aug 06, 2018 11:33 pm

Tosyk wrote:chrrox, thanks for clarify this a bit

so what this line (meshGroupInfo.append(bs.read("9I"))) exactly do? can I read vertex buffer (12 bytes in stride) the same way? like (bs.read("3I"))?

it's unfortune that noesis doesn't have documentation for its python functions nor detailed/explained examples

(meshGroupInfo.append(bs.read("9I"))) just split python functions to make them easier to understand.
1. meshGroupInfo.append()
append whats in the brackets to the list meshGroupInfo
2. bs.read("9I")
bs is he current file stream variable
read("9I")
read 9 ints in a row

so lets do an example

say I have 1 model with 3 meshes
each mesh has the info for
vert count, face count, material id
I would read it like
meshGroupInfo = []
for i in range(0,3):
meshGroupInfo.append(bs.read("3I"))
(mesh data could be here or later in a file)
(We will assume its later so I can call it like this)

for i in range(0,3):
print(meshGroupInfo[0], "vert count")
print(meshGroupInfo[1], "face count")
print(meshGroupInfo[2], "material id")

User avatar
Tosyk
double-veteran
double-veteran
Posts: 934
Joined: Thu Oct 22, 2009 10:24 am
Location: Russia, Siberia
Has thanked: 225 times
Been thanked: 109 times
Contact:

Re: Extracting simple models

Post by Tosyk » Wed Aug 08, 2018 10:32 am

chrrox wrote:meshGroupInfo = []
for i in range(0,3):
meshGroupInfo.append(bs.read("3I"))
(mesh data could be here or later in a file)
(We will assume its later so I can call it like this)
so here you create a mechanism by which you'll read further info. in maxscript word: read 3 floats - one after another
chrrox wrote:for i in range(0,3):
print(meshGroupInfo[0], "vert count")
print(meshGroupInfo[1], "face count")
print(meshGroupInfo[2], "material id")
and here you apply created mechanism to the stride and read: 3 flots for vertex, 3 flots for face, 3 flots for mat ID

AND
these "vert count", "face count", "material id" is multipliers, correct?

edit:
and this code:

Code: Select all

			for j in range(meshNumGroups):
				if fileType == 9 or fileType == 10:
					meshGroupInfo.append(bs.read("9I"))
				elif fileType == 11:
					meshGroupInfo.append(bs.read("10I"))
means a rule for the stride size: for 9 or 10 version size would be 9 floats and for 11 version size would be 10 floats (1 additional float to the normal version). correct?
and maxscript example for that rule would be (right?):

Code: Select all

if vertexSize == 48 then
					(
						for x = 1 to vertexcount do
							(
								getPos = ftell f + vertexSize	
								dx=readFloat f
								dy=readFloat f
								dz=readFloat f
								
								null = readlong f
								
								x3=readHalfFloat f
								x4=readHalfFloat f
								x5=readHalfFloat f
								x6=readHalfFloat f
								
								tu=readHalfFloat f --first UV channel
								tv=readHalfFloat f --first UV channel
								
								x9=readshort f
								x10=readshort f
								x11=readshort f
								x12=readshort f --always the same
								
								bone1 = readbyte f #unsigned
								bone2 = readbyte f #unsigned
								bone3 = readbyte f #unsigned
								bone4 = readbyte f #unsigned	
								weight1 = readbyte f #unsigned
								weight2 = readbyte f #unsigned
								weight3 = readbyte f #unsigned
								weight4 = readbyte f #unsigned
								
								readfloat f
								
								w = (weight_data boneids:#() weights:#())
								maxweight = 0
								if(weight1 != 0) then maxweight = maxweight + weight1
								if(weight2 != 0) then maxweight = maxweight + weight2
								if(weight3 != 0) then maxweight = maxweight + weight3
								if(weight4 != 0) then maxweight = maxweight + weight4

								if(maxweight != 0) then
									(
										if(weight1 != 0) then
											(
												w1 = weight1 as float
												append w.boneids (bone1 + 1)
												append w.weights (w1 / 255.0)
											)

										if(weight2 != 0) then
											(
												w2 = weight2 as float
												append w.boneids (bone2 + 1)
												append w.weights (w2 / 255.0)
											)

										if(weight3 != 0) then
											(
												w3 = weight3 as float
												append w.boneids (bone3 + 1)
												append w.weights (w3 / 255.0)
											)

										if(weight4 != 0) then
											(
												w4 = weight4 as float
												append w.boneids (bone4 + 1)
												append w.weights (w4 / 255.0)
											)
									)

								append Weight_array w			
								fseek f getPos #seek_set
								append Vert_array[dx,dz,dy] 
								append UV_array[tu,tv*-1,0]
							)
					)
Thank you for all you do here
my blog | my forum

thorh62
ultra-n00b
Posts: 3
Joined: Sat Feb 10, 2018 4:45 am
Has thanked: 1 time

Re: Extracting simple models

Post by thorh62 » Thu Aug 09, 2018 8:54 pm

Hello all,

first of all, thanks for providing this valuable tool :) .

I am facing a problem when running the latest version of hex2obj. I get two error messages "wordpad.exe" is missing
and also "glut32.dll" is missing. I am pretty sure that I have wordpad installed in my computer.

Maybe it is because I am using Windows 10?

Any advice about how to fix these errors? unfortunately these errors are preventing me from reading the tutorial (because I guess the tutorial might be using wordpad?) and also .obj files can not be created or previewed because I am missing the second file.

User avatar
shakotay2
MEGAVETERAN
MEGAVETERAN
Posts: 2730
Joined: Fri Apr 20, 2012 9:24 am
Location: Nexus, searching for Jim Kirk
Has thanked: 675 times
Been thanked: 1407 times

Re: Extracting simple models

Post by shakotay2 » Thu Aug 09, 2018 9:59 pm

hex2obj exes are NOT working standalone, so
for zips with exe-only you need the files in this zip as a base, too:

---------- version 0.24e ----------
hex2obj_0.24e.zip

or ---------- >>> version 0.24c <<< ----------
hex2obj_0.24c.zip

Copy the lastest exe to the unzipped folder then.

( btw: wordpad.exe is not required, you can just double click hex2obj_tut.rtf )
Last edited by shakotay2 on Tue Sep 03, 2019 5:31 pm, edited 1 time in total.
Bigchillghost, Reverse Engineering a Game Model: viewtopic.php?f=29&t=17889
extracting simple models: viewtopic.php?f=29&t=10894
Make_H2O-ForzaHor3-jm9.zip
"You quoted the whole thing, what a mess."

User avatar
Tosyk
double-veteran
double-veteran
Posts: 934
Joined: Thu Oct 22, 2009 10:24 am
Location: Russia, Siberia
Has thanked: 225 times
Been thanked: 109 times
Contact:

Re: Extracting simple models

Post by Tosyk » Sun Aug 19, 2018 10:27 am

I have good progress in the script for division. right now I'm trying to understand where Gh0stBlade getting mesh scaler. also I noticed that some meshes are not fit their skeleton just a bit. for example on this screenshot full model need to be scaled down fo 1% and moved by Y:-0.1 to fit the skeleton. I found that some of the head models need to be transformed the same way but with different numbers.

Image

I would apreciate if anyone explain to me how this (viewtopic.php?p=118852#p118852) python reads scaler
Thank you for all you do here
my blog | my forum

Post Reply