XeNTaX Forum Index
Forum MultiEx Commander Tools Tools Home
It is currently Wed Sep 19, 2018 9:10 pm

All times are UTC + 1 hour




Post new topic Reply to topic  [ 385 posts ]  Go to page Previous  1 ... 20, 21, 22, 23, 24, 25, 26  Next
Author Message
 Post subject: Re: Extracting simple models
PostPosted: Thu Jul 05, 2018 2:57 pm 
Offline
M-M-M-Monster veteran
M-M-M-Monster veteran
User avatar

Joined: Fri Apr 20, 2012 9:24 am
Posts: 2376
Location: Nexus, searching for Jim Kirk
Has thanked: 557 times
Have thanks: 1240 times














You can make the ads go away by registering

onelove1210 wrote:
I've done a backup while running a game to get a dump or a cache file from what are on game screen. [...]

What I am looking for is geometry data, and also a header or a range of geometry data so I can cut it out to explore it easier. This game can't be ripped by Ninja Ripper - weird distortions because of encryption (maybe). That's why I would like to see what the default structure of geometry data in this game is since this game archive can't be extracted atm.
It would make sense if you created a new thread for such because this one as you may have noticed is called "Extracting simple models".

_________________
Bigchillghost, Reverse Engineering a Game Model: http://forum.xentax.com/viewtopic.php?f=29&t=17889
extracting simple models: http://forum.xentax.com/viewtopic.php?f=29&t=10894
Make_H2O-ForzaHor3-jm9.zip


Top
 Profile  
 
 Post subject: Re: Extracting simple models
PostPosted: Thu Jul 05, 2018 3:14 pm 
Offline
advanced

Joined: Thu Apr 07, 2011 12:06 pm
Posts: 75
Has thanked: 8 times
Have thanks: 4 times
shakotay2 wrote:
It would make sense if you created a new thread for such because this one as you may have noticed is called "Extracting simple models".


That is what I wondered. Since this file was a bunch of data, not an archive nor geometry-only file, I decided to give it a chance in this thread first. I need your help to know if there were geometry data in that file first. Then if we can figure out the structure of geometry data directly from that bunch, I can get it easier in other files as well as create a new thread with more detailed information :]


Top
 Profile  
 
 Post subject: Re: Extracting simple models
PostPosted: Thu Jul 05, 2018 3:28 pm 
Offline
mega-veteran
mega-veteran
User avatar

Joined: Tue Jul 05, 2016 9:37 am
Posts: 308
Has thanked: 16 times
Have thanks: 162 times
onelove1210 wrote:
I want to know a range or "magic" key (header) of geometry data to search them easier in other files.

How did you backup the game? A memory dump? And are the two files you uploaded before all that you got? What's the game name btw?

_________________
"No investigation means no right to speak."
An Imitable Workflow for Reverse Engineering a Game Model

Don't forget to thank people by clicking the Image when you get helped.


Top
 Profile  
 
 Post subject: Re: Extracting simple models
PostPosted: Thu Jul 05, 2018 3:34 pm 
Offline
advanced

Joined: Thu Apr 07, 2011 12:06 pm
Posts: 75
Has thanked: 8 times
Have thanks: 4 times
Bigchillghost wrote:
How did you backup the game? A memory dump? And are the two files you uploaded before all that you got? What's the game name btw?


No I did a backup for the whole machine. Maybe it contained a memory dump. Game's name is 决战!平安京, I have no idea what is the English name of this game because I'm not Chinese. Actually I got a lot of files but I thought those two were the most useful to figure out geometry data based on what I've seen in Hex Viewer.


Top
 Profile  
 
 Post subject: Re: Extracting simple models
PostPosted: Thu Jul 05, 2018 3:48 pm 
Offline
mega-veteran
mega-veteran
User avatar

Joined: Tue Jul 05, 2016 9:37 am
Posts: 308
Has thanked: 16 times
Have thanks: 162 times
onelove1210 wrote:
No I did a backup for the whole machine. Maybe it contained a memory dump.

Ah, I see.
onelove1210 wrote:
Game's name is 决战!平安京, I have no idea what is the English name of this game because I'm not Chinese.

It's exactly the one in this post:
http://forum.xentax.com/viewtopic.php?f=29&t=17924
and here:
http://forum.xentax.com/viewtopic.php?f=16&t=17930
onelove1210 wrote:
Actually I got a lot of files but I thought those two were the most useful to figure out geometry data based on what I've seen in Hex Viewer.

See if you can get a clue in other files. Try searching a scrambled "alphabet" table like this one:
Image

_________________
"No investigation means no right to speak."
An Imitable Workflow for Reverse Engineering a Game Model

Don't forget to thank people by clicking the Image when you get helped.


Top
 Profile  
 
 Post subject: Re: Extracting simple models
PostPosted: Thu Jul 05, 2018 4:09 pm 
Offline
advanced

Joined: Thu Apr 07, 2011 12:06 pm
Posts: 75
Has thanked: 8 times
Have thanks: 4 times
Bigchillghost wrote:


You can see that there is no solution, and Ninja Ripper is not working :D

Bigchillghost wrote:
See if you can get a clue in other files. Try searching a scrambled "alphabet" table like this one:
....


Ok, I will find it then. That means there is no similar scrambled "alphabet" table in 2 files I uploaded? Anyway thanks for your help :]

Edit: Is this a table you need?

Image


Top
 Profile  
 
 Post subject: Re: Extracting simple models
PostPosted: Thu Jul 05, 2018 4:23 pm 
Offline
mega-veteran
mega-veteran
User avatar

Joined: Tue Jul 05, 2016 9:37 am
Posts: 308
Has thanked: 16 times
Have thanks: 162 times
onelove1210 wrote:
Edit: Is this a table you need?

No. This seems to be an offset table. You're supposed to search for an array of short intergers which are in an increasing order overall, as I descripted in this post:
viewtopic.php?p=139003#p139003

_________________
"No investigation means no right to speak."
An Imitable Workflow for Reverse Engineering a Game Model

Don't forget to thank people by clicking the Image when you get helped.


Top
 Profile  
 
 Post subject: Re: Extracting simple models
PostPosted: Thu Jul 05, 2018 4:33 pm 
Offline
advanced

Joined: Thu Apr 07, 2011 12:06 pm
Posts: 75
Has thanked: 8 times
Have thanks: 4 times
Bigchillghost wrote:
onelove1210 wrote:
Edit: Is this a table you need?

No. This seems to be an offset table. You're supposed to search for an array of short intergers which are in an increasing order overall, as I descripted in this post:
viewtopic.php?p=139003#p139003


Image

This one is an only scrambled alphabet table. I've read your tutorial before. However it's difficult to find that in a bunch of data (~1.5GB) as I'm a newbie. If it's not possible, I think I will give up then :(


Top
 Profile  
 
 Post subject: Re: Extracting simple models
PostPosted: Thu Jul 05, 2018 4:48 pm 
Offline
mega-veteran
mega-veteran
User avatar

Joined: Tue Jul 05, 2016 9:37 am
Posts: 308
Has thanked: 16 times
Have thanks: 162 times
onelove1210 wrote:
However it's difficult to find that in a bunch of data (~1.5GB) as I'm a newbie. If it's not possible, I think I will give up then :(

Well then you can search "00 00 00 01 00 02" and see if there's anything matches it.

_________________
"No investigation means no right to speak."
An Imitable Workflow for Reverse Engineering a Game Model

Don't forget to thank people by clicking the Image when you get helped.


Top
 Profile  
 
 Post subject: Re: Extracting simple models
PostPosted: Thu Jul 05, 2018 5:10 pm 
Offline
advanced

Joined: Thu Apr 07, 2011 12:06 pm
Posts: 75
Has thanked: 8 times
Have thanks: 4 times
Bigchillghost wrote:
Well then you can search "00 00 00 01 00 02" and see if there's anything matches it.


Of course that's what I have done :D Anyway, sorry to disturb you, I'll give it a chance later


Top
 Profile  
 
 Post subject: Re: Extracting simple models
PostPosted: Thu Jul 19, 2018 10:48 pm 
Offline
double-veteran
double-veteran
User avatar

Joined: Thu Oct 22, 2009 10:24 am
Posts: 925
Location: Russia, Siberia
Has thanked: 218 times
Have thanks: 104 times
I have a head mesh. there's a 4 submeshes: eyes, eyelashes, face and teethguard.
I build vertices, polys and uv for all except the face — it splitted into small pieces and each piece separate with some sort of padding usually look like this:
Code:
06 00 00 00 D3 9B 6A 3F
06 00 00 00 B4 A6 6C 3F
06 00 00 00 94 34 6E 3F

probably I can delete all this garbage and build head (called face here) mesh normally but right now I'm trying to automate this process. I'm using Model Researcher.

sample and Model Researcher template:
https://drive.google.com/open?id=15Pqlx ... zMJUDxdfnw

please help me. this is my first attempt

edit: there a lot of morphs, bone names at the end of the file.

etit2: I've noticed that some of paddings between "vertex size" are longer than 4 bytes and could be: 05000000 31EE0D3D 06000000 F2616A3C 07000000 E001753C. since there's are morphs I think this info could be connected to that morphs somehow.

edit3: after more research I found the following. I thought at first that normal "vertex size" long is 88 bytes but it's a bit more complicated. For example I took a block which size is 104 bytes:
3 floats (cyan): for vertex XYZ
9 floats (black): unknow at the moment, possible normals, vertex color, weights etc
4 floats (pale green): it's a usual placeholder for this format, it will be the same for each vertex in this file
1 float (yellow): I believe it stands for UV channel number but unsure
2 floats (pink): UV
1 float (blue): this one is tricky, it defines how many float pares will be after it. I think this float and the ones next to it are some kind of modifiers of this vertex
2-8 floats (pale purple): it could be from 2 to 8 floats, didn't see more. it's a vertex modifiers as I call them. they could be related to the morphs or something else

right now I need to skip this floats after UV section, any tips?


btw how should I call this line of symbols related to single vertex? Vertex block? vertex size long?


Attachments:


You do not have the required permissions to view the files attached to this post. Register to gain access.


_________________
Thank you for all you do here
my blog | my forum


Top
 Profile  
 
 Post subject: Re: Extracting simple models
PostPosted: Fri Jul 20, 2018 7:18 am 
Offline
mega-veteran
mega-veteran
User avatar

Joined: Tue Jul 05, 2016 9:37 am
Posts: 308
Has thanked: 16 times
Have thanks: 162 times
Tosyk wrote:
1 float (blue): this one is tricky, it defines how many float pares will be after it. I think this float and the ones next to it are some kind of modifiers of this vertex
2-8 floats (pale purple): it could be from 2 to 8 floats, didn't see more. it's a vertex modifiers as I call them. they could be related to the morphs or something else

It's nothing special but merely a simple structure for skin info:
Code:
long   boneCount;
for(i = 0; i < boneCount; i++)
{
   long   boneID;
   float   boneWeight
}

_________________
"No investigation means no right to speak."
An Imitable Workflow for Reverse Engineering a Game Model

Don't forget to thank people by clicking the Image when you get helped.


Top
 Profile  
 
 Post subject: Re: Extracting simple models
PostPosted: Fri Jul 20, 2018 10:27 am 
Offline
double-veteran
double-veteran
User avatar

Joined: Thu Oct 22, 2009 10:24 am
Posts: 925
Location: Russia, Siberia
Has thanked: 218 times
Have thanks: 104 times
ahh, thats what 00 00 80 3F stands for it's 100% (red) weight for head bone, am I right?

also, I think I need to create some sort of script or plugin to build mesh out of this data. So any advice on this would be greatfully usefull! Noesis, maxscript? Can I use structure I found right away with noesis python to build the mesh and then put in order this code to load other models correctly?

so, for example, in Model Researcher I can set vertex block start and vertex count to build vertex cloud in a few seconds, can I do the same in noesis to just understand the basic?

btw, this is too complicated at this point of my knowledge :(
Bigchillghost wrote:
It's nothing special but merely a simple structure for skin info:
Code:
long   boneCount;
for(i = 0; i < boneCount; i++)
{
   long   boneID;
   float   boneWeight
}

_________________
Thank you for all you do here
my blog | my forum


Top
 Profile  
 
 Post subject: Re: Extracting simple models
PostPosted: Fri Jul 20, 2018 11:10 am 
Offline
mega-veteran
mega-veteran
User avatar

Joined: Tue Jul 05, 2016 9:37 am
Posts: 308
Has thanked: 16 times
Have thanks: 162 times
Tosyk wrote:
ahh, thats what 00 00 80 3F stands for it's 100% (red) weight for head bone, am I right?

If there's only one bone used by that vert, of course.
Tosyk wrote:
Noesis, maxscript?

Whatever you prefer.
Tosyk wrote:
Can I use structure I found right away with noesis python to build the mesh and then put in order this code to load other models correctly?

If you mean loading a certain model only, you might have already got enough info. But if you wanna automate the process for this format, you need to know how to locate to the next mesh, or you just search for a certain byte sequence which I don't really recommend unless there's no other way.
Tosyk wrote:
so, for example, in Model Researcher I can set vertex block start and vertex count to build vertex cloud in a few seconds, can I do the same in noesis to just understand the basic?

Actually there's not much difference between the two processes. The only difference is that to make an automatic tool your program should be able to obtain necessary info by itself rather than depending on a human mind.
Tosyk wrote:
btw, this is too complicated at this point of my knowledge :(

It's just a piece of pseudo code in C to help you understand the structure of the vertex format. It's equivalent to the following pseudo code.
Code:
Type   Specification

long   boneCount
for i = 1 to boneCount
{
   long   boneID
   float   boneWeight
}

_________________
"No investigation means no right to speak."
An Imitable Workflow for Reverse Engineering a Game Model

Don't forget to thank people by clicking the Image when you get helped.


Top
 Profile  
 
 Post subject: Re: Extracting simple models
PostPosted: Fri Jul 20, 2018 12:28 pm 
Offline
M-M-M-Monster veteran
M-M-M-Monster veteran
User avatar

Joined: Fri Apr 20, 2012 9:24 am
Posts: 2376
Location: Nexus, searching for Jim Kirk
Has thanked: 557 times
Have thanks: 1240 times
Tosyk wrote:
So any advice on this would be greatfully usefull! Noesis, maxscript?
Why not use python with ModelResearcher (MR)? http://lazov.ru/mr/tutorial.php?name=tut_mr_scripts

(I did not use it but if you're using MR it would make sense.)

Anyways, here's a Noesis script to just load ellis_head (the eyeball, to be exact) and nothing more:
Code:
from inc_noesis import *
import noesis
import rapi

def registerNoesisTypes():
    handle = noesis.register("ellis_head", ".wwm")
    noesis.setHandlerTypeCheck(handle, noepyCheckType)
    noesis.setHandlerLoadModel(handle, noepyLoadModel)
    return 1

def noepyCheckType(data):
    bs = NoeBitStream(data)
    idstring = bs.readUInt()
    if idstring != 0x00000049:
        print("not a wwm file!")
        return 0
    return 1

def noepyLoadModel(data, mdlList):   
    ctx = rapi.rpgCreateContext()
    bs = NoeBitStream(data)
    #rapi.rpgClearBufferBinds()
    bs.seek(0x8b420, NOESEEK_ABS)
    size = 26864#= 5037/3*4*4
    idxbuf_tmp = bs.readBytes(size)

    numFaces = 5037//3
    print("numFaces", numFaces)
    idxBuf = []
    for i in range(numFaces):
        idx = idxbuf_tmp[i*16 + 4] + idxbuf_tmp[i*16 + 5]*256
        idxBuf.append(idx)
        idx = idxbuf_tmp[i*16+ 8] + idxbuf_tmp[i*16 + 9]*256
        idxBuf.append(idx)
        idx = idxbuf_tmp[i*16 + 12] + idxbuf_tmp[i*16 + 13]*256
        idxBuf.append(idx)


    #print(idxBuf)

    bs.seek(0x368, NOESEEK_ABS)
    vertsCount = 884
    VertBuf = bs.readBytes(vertsCount * 88)
    b = struct.pack('H' * len(idxBuf), *idxBuf)

    rapi.rpgBindPositionBufferOfs(VertBuf, noesis.RPGEODATA_FLOAT, 88, 0)
    rapi.rpgCommitTriangles(b, noesis.RPGEODATA_USHORT, numFaces*3, noesis.RPGEO_TRIANGLE, 1)

    mdl = rapi.rpgConstructModel()
    mdlList.append(mdl)

    return 1


Very important:
Bigchillghost wrote:
But if you wanna automate the process for this format, you need to know how to locate to the next mesh,


Quote:
or you just search for a certain byte sequence which I don't really recommend unless there's no other way.
well, I wouldn't say so. It's the most recommended way for beginners, imho.


Attachments:


You do not have the required permissions to view the files attached to this post. Register to gain access.


_________________
Bigchillghost, Reverse Engineering a Game Model: http://forum.xentax.com/viewtopic.php?f=29&t=17889
extracting simple models: http://forum.xentax.com/viewtopic.php?f=29&t=10894
Make_H2O-ForzaHor3-jm9.zip


Last edited by shakotay2 on Fri Jul 20, 2018 12:34 pm, edited 1 time in total.

Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 385 posts ]  Go to page Previous  1 ... 20, 21, 22, 23, 24, 25, 26  Next

All times are UTC + 1 hour


Who is online

Users browsing this forum: No registered users and 2 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group