Hello all, we have been targeted by a certain type of spambot that will send messages to our users in their private messages. If you get a message from such a loser/bot, report it to us here, and delete the message !

Extracting simple models

Read or post any tutorial related to file format analysis for modding purposes.
User avatar
shakotay2
MEGAVETERAN
MEGAVETERAN
Posts: 2667
Joined: Fri Apr 20, 2012 9:24 am
Location: Nexus, searching for Jim Kirk
Has thanked: 654 times
Been thanked: 1364 times

Re: Extracting simple models

Post by shakotay2 » Sun May 13, 2018 5:33 pm

Bigchillghost wrote:So you do all the works in every single function combination? Why not just read the vertices/UVs data to a temporary buffer and convert/copy them to the destination buffer? Then you need to process only one buffer if the flipping UVs option is chosen.
well, a temporary buffer doesn't help. I use one buffer only and depending on whether there's half float uvs in there or shorts you choose (function) HF_UV or (function) WordUV from the dropdown combobox. The "knowledge" how to treat the values in the buffer is contained in those function. (It was a bad architectural design decision, I know, but it can't be corrected by simple means, imho.)
BTW I'm still confused about the UI. What are the w/64k / w/8k and the grey CF buttons for?
CF is combined face indices, not implemented.
The w/x is active for WordUV and ShortAll only as you may have noticed, it's just the divider for word (short) uvs because there were formats where the uv map appeared "zoomed out".
Also, I noticed that the exported mesh is grouped per 500 triangles, is it because the face buffer is that big, or that it's a designed feature? I wish I don't have to blocked them out every time.
It's for bigger meshes where the submesh borders are hard to find.
In blender wavefront importer there's the option to load Split by Group.
If you uncheck that feature it's remains for all mehses loaded afterwards still you check it again or restart blender.

(I didn't think about other 3D apps, so sorry for that.)
Bigchillghost, Reverse Engineering a Game Model: viewtopic.php?f=29&t=17889
extracting simple models: viewtopic.php?f=29&t=10894
Make_H2O-ForzaHor3-jm9.zip
"You quoted the whole thing, what a mess."

User avatar
Bigchillghost
ultra-veteran
ultra-veteran
Posts: 432
Joined: Tue Jul 05, 2016 9:37 am
Has thanked: 21 times
Been thanked: 296 times

Re: Extracting simple models

Post by Bigchillghost » Mon May 14, 2018 6:06 am

shakotay2 wrote:The "knowledge" how to treat the values in the buffer is contained in those function. (It was a bad architectural design decision, I know, but it can't be corrected by simple means, imho.)
I'm not sure if I understand correctly, but if the buffer is allocated and freed in these functions, I guess there'd be a lot of work to do. Maybe you should consider rewrite these functions and preserve only the data type convertion parts. It shouldn't be too hard to do though.
CF is combined face indices, not implemented.
The w/x is active for WordUV and ShortAll only as you may have noticed, it's just the divider for word (short) uvs because there were formats where the uv map appeared "zoomed out".
Ah, I see. Thanks for the clarification!
I didn't think about other 3D apps, so sorry for that.
Actually 3Ds Max has similar option but I usually leave it unchecked incase the order of the grouped submeshes is changed.
Just need 2 steps to solve it in Notepad++ anyway.
May you find peace in this puzzle-solving game. Say it with action: click the Image when you get helped.:)

User avatar
shakotay2
MEGAVETERAN
MEGAVETERAN
Posts: 2667
Joined: Fri Apr 20, 2012 9:24 am
Location: Nexus, searching for Jim Kirk
Has thanked: 654 times
Been thanked: 1364 times

Re: Extracting simple models

Post by shakotay2 » Mon May 14, 2018 7:46 am

Bigchillghost wrote:I'm not sure if I understand correctly, but if the buffer is allocated and freed in these functions, I guess there'd be a lot of work to do. Maybe you should consider rewrite these functions and preserve only the data type convertion parts.
The buffer is static and freed at the end of the application.

"rewrite"?, well, I consider hex2obj as "finished", more or less. I think my coding time is better invested into the Make_obj project (with source, partially from hex2obj) or the "Skeleton to SMD" project which I hopefully get released 'til year's end. :D
Bigchillghost, Reverse Engineering a Game Model: viewtopic.php?f=29&t=17889
extracting simple models: viewtopic.php?f=29&t=10894
Make_H2O-ForzaHor3-jm9.zip
"You quoted the whole thing, what a mess."

User avatar
Bigchillghost
ultra-veteran
ultra-veteran
Posts: 432
Joined: Tue Jul 05, 2016 9:37 am
Has thanked: 21 times
Been thanked: 296 times

Re: Extracting simple models

Post by Bigchillghost » Mon May 14, 2018 10:42 am

shakotay2 wrote:The buffer is static and freed at the end of the application.
I think I know what you mean now. The files test.obj and testUVs.obj are generated in those functions whereas the exported obj file is created from the static buffer after being processed by these functions, am I right? Well then you can flip the UVs for the exported obj file only and I think the users should know what they're doing.
shakotay2 wrote:"rewrite"?, well, I consider hex2obj as "finished", more or less.
Yeah, I hate to rework on some of my old projects as well but still I wouldn't hesitate to make some improvements if possible.
shakotay2 wrote:I think my coding time is better invested into the Make_obj project (with source, partially from hex2obj)
Well, about that, I really wonder what this project is for. :D I mean, if you know how to programing, why not write a full-featured obj convertor instead? You can even make full use of the normals data, which is far more accurate than the one auto-generated by the 3D softwares. So with all due respect, maybe you're the only advanced user of this project, while the rest, are those who have little or no experience in programing and reverse engineering a game model.
shakotay2 wrote:or the "Skeleton to SMD" project which I hopefully get released 'til year's end. :D
Is that something similar to hex2obj? If so it would definitely be the most anticipated tool for reverse engineering skeleton data, and also the first one.:D

I have some basic understanding on the bone stuffs now, including bone names, bone IDs, parent bone IDs, bone indices, bone weights, and bone matrices, which is still the trickiest part to me. I do know there're translations, rotations, scales, maybe even projections in every matrix though. Thing is that I don't know how to organize these data into a usable format. Would you consider writting a tutorial on related stuffs by that time?
May you find peace in this puzzle-solving game. Say it with action: click the Image when you get helped.:)

User avatar
shakotay2
MEGAVETERAN
MEGAVETERAN
Posts: 2667
Joined: Fri Apr 20, 2012 9:24 am
Location: Nexus, searching for Jim Kirk
Has thanked: 654 times
Been thanked: 1364 times

Re: Extracting simple models

Post by shakotay2 » Mon May 14, 2018 11:21 pm

Bigchillghost wrote:Well then you can flip the UVs for the exported obj file only
The buffer itself remains unchanged. Here's the float version for uvs from the Make_obj project. The flipping would have to be applied in any of those functions:

Code: Select all

void logUVs(DWORD addrUV, DWORD Vcnt, BYTE vStride)
{
    char *pFBuf ;
    BYTE i ;
    DWORD k ;
    float *pFloat ;
	float fData = 0.0f;

    pFloat= &fData ;
    pFBuf = (char *) lpFBuf ;
    pFBuf += addrUV ;
	for (k=0;k < Vcnt;k++) {
        fprintf( stream, "vt ") ;
        for (i=0;i<2;i++) {
            pFloat = (float*) pFBuf ; fprintf( stream, "%f ", *pFloat) ; pFBuf += 4 ;
        }
        fprintf( stream, "\n") ;
		pFBuf += vStride-8 ;
	}
}
Well, about that, I really wonder what this project is for. :D
It can't be addressed to the majority of forum members, I'm aware of that. It's intended to give a handful of newcomers an idea how to start.
I mean, if you know how to programing, why not write a full-featured obj convertor instead?
Why should I do? This is just a hobby of mine and "full-featured" means "time eating". :D
Thing is that I don't know how to organize these data into a usable format.
yeah, that's the main problem. And there's so many variations which will lead to confusions of the customers, more than with hex2obj (which is known as the tool where "you have to input random numbers" :D ). In the end I'm not sure whether it's a good idea to release tools which are primary designed as helper tools for my personal use.
Would you consider writting a tutorial on related stuffs by that time?
There will be a short tutorial on the use of skel2smd. But I'm not going into details (because of lack of knowledge). Indeed I hope to find some "basic data types" which the user can toggle through.
Bigchillghost, Reverse Engineering a Game Model: viewtopic.php?f=29&t=17889
extracting simple models: viewtopic.php?f=29&t=10894
Make_H2O-ForzaHor3-jm9.zip
"You quoted the whole thing, what a mess."

User avatar
Bigchillghost
ultra-veteran
ultra-veteran
Posts: 432
Joined: Tue Jul 05, 2016 9:37 am
Has thanked: 21 times
Been thanked: 296 times

Re: Extracting simple models

Post by Bigchillghost » Tue May 15, 2018 6:20 am

shakotay2 wrote: The buffer itself remains unchanged.
Oops...
shakotay2 wrote:Why should I do? This is just a hobby of mine and "full-featured" means "time eating". :D
You know, I was actually talking about the target users. And trust me, it won't save you much time by gathering only the offsets and the counts. But regarding this:
shakotay2 wrote: It's intended to give a handful of newcomers an idea how to start.
never mind!
shakotay2 wrote: In the end I'm not sure whether it's a good idea to release tools which are primary designed as helper tools for my personal use.
Well I'm sure such tools are meant for users who actually know what they're doing. For those who don't, they just need some more basic knowledge.
May you find peace in this puzzle-solving game. Say it with action: click the Image when you get helped.:)

onelove1210
advanced
Posts: 75
Joined: Thu Apr 07, 2011 12:06 pm
Has thanked: 8 times
Been thanked: 4 times

Re: Extracting simple models

Post by onelove1210 » Thu Jul 05, 2018 7:26 am

Hi, I would like to ask if any solution to figure out that mesh or bone data in a file which is created from merged files.
My backup file contains some keywords like "biped" or "bone" in Hex Viewer, however I'm not sure if it's correct mesh or skeleton format, since that file is a combination of many files. Hope you can help me to extract it, or at least find a header and the end of mesh data only. It would be appreciated :D
Last edited by onelove1210 on Fri Jul 06, 2018 3:45 am, edited 1 time in total.

onelove1210
advanced
Posts: 75
Joined: Thu Apr 07, 2011 12:06 pm
Has thanked: 8 times
Been thanked: 4 times

Re: Extracting simple models

Post by onelove1210 » Thu Jul 05, 2018 7:41 am

--double post--

User avatar
Bigchillghost
ultra-veteran
ultra-veteran
Posts: 432
Joined: Tue Jul 05, 2016 9:37 am
Has thanked: 21 times
Been thanked: 296 times

Re: Extracting simple models

Post by Bigchillghost » Thu Jul 05, 2018 8:01 am

onelove1210 wrote:My backup file contains some keywords like "biped" or "bone" in Hex Viewer, however I'm not sure if it's correct mesh or skeleton format, since that file is a combination of many files. Hope you can help me to extract it, or at least find a header and the end of mesh data only. It would be appreciated :D
There's no geometry data inside your file but merely some RIFF audio.
May you find peace in this puzzle-solving game. Say it with action: click the Image when you get helped.:)

onelove1210
advanced
Posts: 75
Joined: Thu Apr 07, 2011 12:06 pm
Has thanked: 8 times
Been thanked: 4 times

Re: Extracting simple models

Post by onelove1210 » Thu Jul 05, 2018 9:31 am

Bigchillghost wrote:
onelove1210 wrote:My backup file contains some keywords like "biped" or "bone" in Hex Viewer, however I'm not sure if it's correct mesh or skeleton format, since that file is a combination of many files. Hope you can help me to extract it, or at least find a header and the end of mesh data only. It would be appreciated :D
There's no geometry data inside your file but merely some RIFF audio.
-----edited------
Last edited by onelove1210 on Fri Jul 06, 2018 3:45 am, edited 1 time in total.

User avatar
Bigchillghost
ultra-veteran
ultra-veteran
Posts: 432
Joined: Tue Jul 05, 2016 9:37 am
Has thanked: 21 times
Been thanked: 296 times

Re: Extracting simple models

Post by Bigchillghost » Thu Jul 05, 2018 10:15 am

onelove1210 wrote:or at least skeleton data?
I wouldn't place "at least" before "skeleton" but no, nothing much make sense to me. I think you should put it clear first what you're looking for, else even if there's geo data there it might not be what you want. And where are these files from? A custom game backup?
May you find peace in this puzzle-solving game. Say it with action: click the Image when you get helped.:)

onelove1210
advanced
Posts: 75
Joined: Thu Apr 07, 2011 12:06 pm
Has thanked: 8 times
Been thanked: 4 times

Re: Extracting simple models

Post by onelove1210 » Thu Jul 05, 2018 10:32 am

Bigchillghost wrote: I wouldn't place "at least" before "skeleton" but no, nothing much make sense to me. I think you should put it clear first what you're looking for, else even if there's geo data there it might not be what you want. And where are these files from? A custom game backup?
I've done a backup while running a game to get a dump or a cache file from what are on game screen. I'm used to seeing a model file which has both of geometry and bone data, so that when I saw "biped" or "bone" , I thought there would be geometry data (Maybe I was wrong).

What I am looking for is geometry data, and also a header or a range of geometry data so I can cut it out to explore it easier. This game can't be ripped by Ninja Ripper - weird distortions because of encryption (maybe). That's why I would like to see what the default structure of geometry data in this game is since this game archive can't be extracted atm.

User avatar
Bigchillghost
ultra-veteran
ultra-veteran
Posts: 432
Joined: Tue Jul 05, 2016 9:37 am
Has thanked: 21 times
Been thanked: 296 times

Re: Extracting simple models

Post by Bigchillghost » Thu Jul 05, 2018 1:04 pm

onelove1210 wrote:I'm used to seeing a model file which has both of geometry and bone data, so that when I saw "biped" or "bone" , I thought there would be geometry data (Maybe I was wrong).
Alway detect the geometry data first, then deal with the bones if you have extra skills.
onelove1210 wrote:What I am looking for is geometry data
This, is a simple answer to the wrong question. You are supposed to specify which character/item you're looking for. There could be some irrelevant model data in the file while searching for them in a bunch of data is just a waste of time.
onelove1210 wrote:That's why I would like to see what the default structure of geometry data in this game is since this game archive can't be extracted atm.
The archives are encrypted? If not it'd be better to work directly with the asset files.
May you find peace in this puzzle-solving game. Say it with action: click the Image when you get helped.:)

onelove1210
advanced
Posts: 75
Joined: Thu Apr 07, 2011 12:06 pm
Has thanked: 8 times
Been thanked: 4 times

Re: Extracting simple models

Post by onelove1210 » Thu Jul 05, 2018 2:52 pm

Bigchillghost wrote:Alway detect the geometry data first, then deal with the bones if you have extra skills.
Just an excuse of thinking bones and meshes always stick together, never mind !
Bigchillghost wrote:This, is a simple answer to the wrong question. You are supposed to specify which character/item you're looking for. There could be some irrelevant model data in the file while searching for them in a bunch of data is just a waste of time.
The archives are encrypted? If not it'd be better to work directly with the asset files.
Oh don't get me wrong. What I mean is not a specific character or item. However I expected there had been one female character and 2 smaller meshes in that file. Like I said above, archives are encrypted (no solution atm) + Ninja Ripper is not working with this game. And this is my full answer above:
onelove1210 wrote:What I am looking for is geometry data, and also a header or a range of geometry data so I can cut it out to explore it easier.
and
onelove1210 wrote:That's why I would like to see what the default structure of geometry data is
Like you said, searching them in a bunch of data is time consuming, that's why I want to know a range or "magic" key (header) of geometry data to search them easier in other files. Meshes from a running game are always in their original structure without encryption or compression, so I think it's good to know what their original format is :D

Since I don't have enough experience to figure out geometry data, I need help from experts :] Hope that makes sense
Last edited by onelove1210 on Thu Jul 05, 2018 3:04 pm, edited 3 times in total.

User avatar
shakotay2
MEGAVETERAN
MEGAVETERAN
Posts: 2667
Joined: Fri Apr 20, 2012 9:24 am
Location: Nexus, searching for Jim Kirk
Has thanked: 654 times
Been thanked: 1364 times

Re: Extracting simple models

Post by shakotay2 » Thu Jul 05, 2018 2:57 pm

onelove1210 wrote:I've done a backup while running a game to get a dump or a cache file from what are on game screen. [...]

What I am looking for is geometry data, and also a header or a range of geometry data so I can cut it out to explore it easier. This game can't be ripped by Ninja Ripper - weird distortions because of encryption (maybe). That's why I would like to see what the default structure of geometry data in this game is since this game archive can't be extracted atm.
It would make sense if you created a new thread for such because this one as you may have noticed is called "Extracting simple models".
Bigchillghost, Reverse Engineering a Game Model: viewtopic.php?f=29&t=17889
extracting simple models: viewtopic.php?f=29&t=10894
Make_H2O-ForzaHor3-jm9.zip
"You quoted the whole thing, what a mess."

Post Reply