READ THE RULES: Click here

Follow us on Facebook: https://www.facebook.com/xentax/ :)

Extracting simple models

Read or post any tutorial related to file format analysis for modding purposes.
onelove1210
advanced
Posts: 75
Joined: Thu Apr 07, 2011 12:06 pm
Has thanked: 8 times
Been thanked: 4 times

Re: Extracting simple models

Post by onelove1210 » Thu Jul 05, 2018 3:14 pm

shakotay2 wrote:It would make sense if you created a new thread for such because this one as you may have noticed is called "Extracting simple models".
That is what I wondered. Since this file was a bunch of data, not an archive nor geometry-only file, I decided to give it a chance in this thread first. I need your help to know if there were geometry data in that file first. Then if we can figure out the structure of geometry data directly from that bunch, I can get it easier in other files as well as create a new thread with more detailed information :]

User avatar
Bigchillghost
ultra-veteran
ultra-veteran
Posts: 501
Joined: Tue Jul 05, 2016 9:37 am
Has thanked: 22 times
Been thanked: 423 times

Re: Extracting simple models

Post by Bigchillghost » Thu Jul 05, 2018 3:28 pm

onelove1210 wrote:I want to know a range or "magic" key (header) of geometry data to search them easier in other files.
How did you backup the game? A memory dump? And are the two files you uploaded before all that you got? What's the game name btw?
May you find peace in this puzzle-solving game. Say it with action: click the Image when you get helped.:)

onelove1210
advanced
Posts: 75
Joined: Thu Apr 07, 2011 12:06 pm
Has thanked: 8 times
Been thanked: 4 times

Re: Extracting simple models

Post by onelove1210 » Thu Jul 05, 2018 3:34 pm

Bigchillghost wrote: How did you backup the game? A memory dump? And are the two files you uploaded before all that you got? What's the game name btw?
No I did a backup for the whole machine. Maybe it contained a memory dump. Game's name is 决战!平安京, I have no idea what is the English name of this game because I'm not Chinese. Actually I got a lot of files but I thought those two were the most useful to figure out geometry data based on what I've seen in Hex Viewer.

User avatar
Bigchillghost
ultra-veteran
ultra-veteran
Posts: 501
Joined: Tue Jul 05, 2016 9:37 am
Has thanked: 22 times
Been thanked: 423 times

Re: Extracting simple models

Post by Bigchillghost » Thu Jul 05, 2018 3:48 pm

onelove1210 wrote:No I did a backup for the whole machine. Maybe it contained a memory dump.
Ah, I see.
onelove1210 wrote:Game's name is 决战!平安京, I have no idea what is the English name of this game because I'm not Chinese.
It's exactly the one in this post:
viewtopic.php?f=29&t=17924
and here:
viewtopic.php?f=16&t=17930
onelove1210 wrote:Actually I got a lot of files but I thought those two were the most useful to figure out geometry data based on what I've seen in Hex Viewer.
See if you can get a clue in other files. Try searching a scrambled "alphabet" table like this one:
Image
May you find peace in this puzzle-solving game. Say it with action: click the Image when you get helped.:)

onelove1210
advanced
Posts: 75
Joined: Thu Apr 07, 2011 12:06 pm
Has thanked: 8 times
Been thanked: 4 times

Re: Extracting simple models

Post by onelove1210 » Thu Jul 05, 2018 4:09 pm

Bigchillghost wrote: It's exactly the one in this post:
viewtopic.php?f=29&t=17924
and here:
viewtopic.php?f=16&t=17930
You can see that there is no solution, and Ninja Ripper is not working :D
Bigchillghost wrote:See if you can get a clue in other files. Try searching a scrambled "alphabet" table like this one:
....
Ok, I will find it then. That means there is no similar scrambled "alphabet" table in 2 files I uploaded? Anyway thanks for your help :]

Edit: Is this a table you need?

Image

User avatar
Bigchillghost
ultra-veteran
ultra-veteran
Posts: 501
Joined: Tue Jul 05, 2016 9:37 am
Has thanked: 22 times
Been thanked: 423 times

Re: Extracting simple models

Post by Bigchillghost » Thu Jul 05, 2018 4:23 pm

onelove1210 wrote: Edit: Is this a table you need?
No. This seems to be an offset table. You're supposed to search for an array of short intergers which are in an increasing order overall, as I descripted in this post:
viewtopic.php?p=139003#p139003
May you find peace in this puzzle-solving game. Say it with action: click the Image when you get helped.:)

onelove1210
advanced
Posts: 75
Joined: Thu Apr 07, 2011 12:06 pm
Has thanked: 8 times
Been thanked: 4 times

Re: Extracting simple models

Post by onelove1210 » Thu Jul 05, 2018 4:33 pm

Bigchillghost wrote:
onelove1210 wrote: Edit: Is this a table you need?
No. This seems to be an offset table. You're supposed to search for an array of short intergers which are in an increasing order overall, as I descripted in this post:
viewtopic.php?p=139003#p139003
Image

This one is an only scrambled alphabet table. I've read your tutorial before. However it's difficult to find that in a bunch of data (~1.5GB) as I'm a newbie. If it's not possible, I think I will give up then :(

User avatar
Bigchillghost
ultra-veteran
ultra-veteran
Posts: 501
Joined: Tue Jul 05, 2016 9:37 am
Has thanked: 22 times
Been thanked: 423 times

Re: Extracting simple models

Post by Bigchillghost » Thu Jul 05, 2018 4:48 pm

onelove1210 wrote:However it's difficult to find that in a bunch of data (~1.5GB) as I'm a newbie. If it's not possible, I think I will give up then :(
Well then you can search "00 00 00 01 00 02" and see if there's anything matches it.
May you find peace in this puzzle-solving game. Say it with action: click the Image when you get helped.:)

onelove1210
advanced
Posts: 75
Joined: Thu Apr 07, 2011 12:06 pm
Has thanked: 8 times
Been thanked: 4 times

Re: Extracting simple models

Post by onelove1210 » Thu Jul 05, 2018 5:10 pm

Bigchillghost wrote:Well then you can search "00 00 00 01 00 02" and see if there's anything matches it.
Of course that's what I have done :D Anyway, sorry to disturb you, I'll give it a chance later

User avatar
Tosyk
double-veteran
double-veteran
Posts: 934
Joined: Thu Oct 22, 2009 10:24 am
Location: Russia, Siberia
Has thanked: 225 times
Been thanked: 110 times
Contact:

Re: Extracting simple models

Post by Tosyk » Thu Jul 19, 2018 10:48 pm

I have a head mesh. there's a 4 submeshes: eyes, eyelashes, face and teethguard.
I build vertices, polys and uv for all except the face — it splitted into small pieces and each piece separate with some sort of padding usually look like this:

Code: Select all

06 00 00 00 D3 9B 6A 3F
06 00 00 00 B4 A6 6C 3F
06 00 00 00 94 34 6E 3F
probably I can delete all this garbage and build head (called face here) mesh normally but right now I'm trying to automate this process. I'm using Model Researcher.

sample and Model Researcher template:
https://drive.google.com/open?id=15Pqlx ... zMJUDxdfnw

please help me. this is my first attempt

edit: there a lot of morphs, bone names at the end of the file.

etit2: I've noticed that some of paddings between "vertex size" are longer than 4 bytes and could be: 05000000 31EE0D3D 06000000 F2616A3C 07000000 E001753C. since there's are morphs I think this info could be connected to that morphs somehow.

edit3: after more research I found the following. I thought at first that normal "vertex size" long is 88 bytes but it's a bit more complicated. For example I took a block which size is 104 bytes:
3 floats (cyan): for vertex XYZ
9 floats (black): unknow at the moment, possible normals, vertex color, weights etc
4 floats (pale green): it's a usual placeholder for this format, it will be the same for each vertex in this file
1 float (yellow): I believe it stands for UV channel number but unsure
2 floats (pink): UV
1 float (blue): this one is tricky, it defines how many float pares will be after it. I think this float and the ones next to it are some kind of modifiers of this vertex
2-8 floats (pale purple): it could be from 2 to 8 floats, didn't see more. it's a vertex modifiers as I call them. they could be related to the morphs or something else

right now I need to skip this floats after UV section, any tips?
b_ellis_head_sample.jpg
btw how should I call this line of symbols related to single vertex? Vertex block? vertex size long?
You do not have the required permissions to view the files attached to this post.
Thank you for all you do here
my blog | my forum

User avatar
Bigchillghost
ultra-veteran
ultra-veteran
Posts: 501
Joined: Tue Jul 05, 2016 9:37 am
Has thanked: 22 times
Been thanked: 423 times

Re: Extracting simple models

Post by Bigchillghost » Fri Jul 20, 2018 7:18 am

Tosyk wrote: 1 float (blue): this one is tricky, it defines how many float pares will be after it. I think this float and the ones next to it are some kind of modifiers of this vertex
2-8 floats (pale purple): it could be from 2 to 8 floats, didn't see more. it's a vertex modifiers as I call them. they could be related to the morphs or something else
It's nothing special but merely a simple structure for skin info:

Code: Select all

long	boneCount;
for(i = 0; i < boneCount; i++)
{
	long	boneID;
	float	boneWeight
}
May you find peace in this puzzle-solving game. Say it with action: click the Image when you get helped.:)

User avatar
Tosyk
double-veteran
double-veteran
Posts: 934
Joined: Thu Oct 22, 2009 10:24 am
Location: Russia, Siberia
Has thanked: 225 times
Been thanked: 110 times
Contact:

Re: Extracting simple models

Post by Tosyk » Fri Jul 20, 2018 10:27 am

ahh, thats what 00 00 80 3F stands for it's 100% (red) weight for head bone, am I right?

also, I think I need to create some sort of script or plugin to build mesh out of this data. So any advice on this would be greatfully usefull! Noesis, maxscript? Can I use structure I found right away with noesis python to build the mesh and then put in order this code to load other models correctly?

so, for example, in Model Researcher I can set vertex block start and vertex count to build vertex cloud in a few seconds, can I do the same in noesis to just understand the basic?

btw, this is too complicated at this point of my knowledge :(
Bigchillghost wrote:It's nothing special but merely a simple structure for skin info:

Code: Select all

long	boneCount;
for(i = 0; i < boneCount; i++)
{
	long	boneID;
	float	boneWeight
}
Thank you for all you do here
my blog | my forum

User avatar
Bigchillghost
ultra-veteran
ultra-veteran
Posts: 501
Joined: Tue Jul 05, 2016 9:37 am
Has thanked: 22 times
Been thanked: 423 times

Re: Extracting simple models

Post by Bigchillghost » Fri Jul 20, 2018 11:10 am

Tosyk wrote:ahh, thats what 00 00 80 3F stands for it's 100% (red) weight for head bone, am I right?
If there's only one bone used by that vert, of course.
Tosyk wrote:Noesis, maxscript?
Whatever you prefer.
Tosyk wrote:Can I use structure I found right away with noesis python to build the mesh and then put in order this code to load other models correctly?
If you mean loading a certain model only, you might have already got enough info. But if you wanna automate the process for this format, you need to know how to locate to the next mesh, or you just search for a certain byte sequence which I don't really recommend unless there's no other way.
Tosyk wrote:so, for example, in Model Researcher I can set vertex block start and vertex count to build vertex cloud in a few seconds, can I do the same in noesis to just understand the basic?
Actually there's not much difference between the two processes. The only difference is that to make an automatic tool your program should be able to obtain necessary info by itself rather than depending on a human mind.
Tosyk wrote:btw, this is too complicated at this point of my knowledge :(
It's just a piece of pseudo code in C to help you understand the structure of the vertex format. It's equivalent to the following pseudo code.

Code: Select all

Type	Specification

long	boneCount
for i = 1 to boneCount
{
	long	boneID
	float	boneWeight
}
May you find peace in this puzzle-solving game. Say it with action: click the Image when you get helped.:)

User avatar
shakotay2
MEGAVETERAN
MEGAVETERAN
Posts: 2767
Joined: Fri Apr 20, 2012 9:24 am
Location: Nexus, searching for Jim Kirk
Has thanked: 687 times
Been thanked: 1426 times

Re: Extracting simple models

Post by shakotay2 » Fri Jul 20, 2018 12:28 pm

Tosyk wrote:So any advice on this would be greatfully usefull! Noesis, maxscript?
Why not use python with ModelResearcher (MR)? http://lazov.ru/mr/tutorial.php?name=tut_mr_scripts

(I did not use it but if you're using MR it would make sense.)

Anyways, here's a Noesis script to just load ellis_head (the eyeball, to be exact) and nothing more:

Code: Select all

from inc_noesis import *
import noesis
import rapi

def registerNoesisTypes():
    handle = noesis.register("ellis_head", ".wwm")
    noesis.setHandlerTypeCheck(handle, noepyCheckType)
    noesis.setHandlerLoadModel(handle, noepyLoadModel)
    return 1

def noepyCheckType(data):
    bs = NoeBitStream(data)
    idstring = bs.readUInt()
    if idstring != 0x00000049:
        print("not a wwm file!")
        return 0
    return 1

def noepyLoadModel(data, mdlList):   
    ctx = rapi.rpgCreateContext()
    bs = NoeBitStream(data)
    #rapi.rpgClearBufferBinds()
    bs.seek(0x8b420, NOESEEK_ABS)
    size = 26864#= 5037/3*4*4
    idxbuf_tmp = bs.readBytes(size)

    numFaces = 5037//3
    print("numFaces", numFaces)
    idxBuf = []
    for i in range(numFaces):
        idx = idxbuf_tmp[i*16 + 4] + idxbuf_tmp[i*16 + 5]*256
        idxBuf.append(idx)
        idx = idxbuf_tmp[i*16+ 8] + idxbuf_tmp[i*16 + 9]*256
        idxBuf.append(idx)
        idx = idxbuf_tmp[i*16 + 12] + idxbuf_tmp[i*16 + 13]*256
        idxBuf.append(idx)


    #print(idxBuf)

    bs.seek(0x368, NOESEEK_ABS)
    vertsCount = 884
    VertBuf = bs.readBytes(vertsCount * 88)
    b = struct.pack('H' * len(idxBuf), *idxBuf)

    rapi.rpgBindPositionBufferOfs(VertBuf, noesis.RPGEODATA_FLOAT, 88, 0)
    rapi.rpgCommitTriangles(b, noesis.RPGEODATA_USHORT, numFaces*3, noesis.RPGEO_TRIANGLE, 1)

    mdl = rapi.rpgConstructModel()
    mdlList.append(mdl)

    return 1
ellis_head(eyeballs)_Noesis.jpg
Very important:
Bigchillghost wrote:But if you wanna automate the process for this format, you need to know how to locate to the next mesh,
or you just search for a certain byte sequence which I don't really recommend unless there's no other way.
well, I wouldn't say so. It's the most recommended way for beginners, imho.
You do not have the required permissions to view the files attached to this post.
Last edited by shakotay2 on Fri Jul 20, 2018 12:34 pm, edited 1 time in total.
Bigchillghost, Reverse Engineering a Game Model: viewtopic.php?f=29&t=17889
extracting simple models: viewtopic.php?f=29&t=10894
Make_H2O-ForzaHor3-jm9.zip
"You quoted the whole thing, what a mess."

User avatar
Tosyk
double-veteran
double-veteran
Posts: 934
Joined: Thu Oct 22, 2009 10:24 am
Location: Russia, Siberia
Has thanked: 225 times
Been thanked: 110 times
Contact:

Re: Extracting simple models

Post by Tosyk » Fri Jul 20, 2018 12:34 pm

Bigchillghost wrote:It's just a piece of pseudo code in C to help you understand the structure of the vertex format. It's equivalent to the following pseudo code.

Code: Select all

Type	Specification

long	boneCount
for i = 1 to boneCount
{
	long	boneID
	float	boneWeight
}
but why you used 'long' for boneCount and boneID (?) — it's 4 bytes each so it should be float, no?

I don't understand all the terminology atm. I know that flot stands for 4 bytes, and half-float is 2 bytes, that's why I used 'float' to define what I found.
alos, how should I call this line of symbols related to single vertex? Vertex block? vertex size long?
shakotay2 wrote:
Tosyk wrote:So any advice on this would be greatfully usefull! Noesis, maxscript?
Why not use python with ModelResearcher (MR)? http://lazov.ru/mr/tutorial.php?name=tut_mr_scripts
wow, I didn't know that, thanks for pointing me :)
shakotay2 wrote:Anyways, here's a Noesis script to just load ellis_head (the eyeball, to be exact) and nothing more
oh, thanks a lot. that's a good start to understand basics :mrgreen:
Thank you for all you do here
my blog | my forum

Post Reply