## Extracting simple models

Read or post any tutorial related to file format analysis for modding purposes.
shakotay2
MEGAVETERAN
Posts: 2605
Joined: Fri Apr 20, 2012 9:24 am
Location: Nexus, searching for Jim Kirk
Has thanked: 638 times
Been thanked: 1332 times

### Extracting simple models

o noobs' hex to obj conversion (poor knowledge required)

It should allow u to extract 3D models with simple formats.
Means such games like Mafia2 or SleepingDogs not to be supported.

- preliminary -

Ok, the truth is - there are two things you should understand:

the hexadecimal numeral system (base 16) using the "numbers" 0..9, A..F
So 0x123 is 291 decimal.
(To avoid confusion with other numbering systems the hex numbers are marked with 0x or an 'h'.)

Secondly you should be able to use a hex editor.

If you don't have these two skills leave now, and come back as soon as you gained them.

(If you don't have a clue which hex editor to use: TinyHexer or HexEdit by A.W. Phillips should do the trick.)
--------------------------------------

As I said you don't need to learn "basics" but you'll have to understand
"the structures" to use hex2obj.

o So here is a short tutorial on it's use.

You don't need to know the technical terms - I just use them habitual.

Simple 3D models consist of vertices and faces.

hex2obj needs the following infos to perform it's task:
the startaddress of the vertices and the one of the faceindices.

(Some experience is required to find them but it's not too hard.)

Secondly we need the vertices count and the faceindices count.

I will explain this with a model example soon but now for the structures -

the two most common are:
a) "sequential"

Code: Select all

``````v x,y,z
v x,y,z
v x,y,z
v x,y,z
...
vn x,y,z
vn x,y,z
vn x,y,z
vn x,y,z
...
vt x,y
vt x,y
vt x,y
vt x,y
...``````
(v: vertex position, vn: normals, vt: uv)
With a sequential model you only need to provide the above mentioned counts.

b) "blocked"

Code: Select all

``````v  x,y,z
vn x,y,z
vt x,y

v  x,y,z
vn x,y,z
vt x,y
...``````
With a blocked model you need to chose the vertex block size (FVF size), for example 32.
(And 'yes': the order might be v, vt, vn for example)

If you don't understand anything you'll have to do a trial 'n error.

Or you calculate the FVF size: each coordinate (x, y or z) consists of 4 bytes.
So the above two blocks have a blocksize of 32 (8x4) each.

It might be a little bit more difficult if there are DWORDs contained in the blocks.
But anyway the most common blocksizes are 32, 40, 52. Sometimes 24, 44, 48 whatever.

o the v,vn,vt patterns to search for

well, to identify what is v, vn or vt is the tricky part. This is for little endian
(for big endian the patterns are in reverse order of course).

v: xx xx xx nn with nn= 42..44 (these are rough values only, can also be 3F, 40)
vn: value of each component 0.0 .. 1.0 (1.0 = 00 00 80 3F)
square sum of components is 1.0 (autocalculated by hex2obj)
vt: value of each component 0.0 .. 1.0

It's important to know that half floats use two bytes only.

------------------------------------------------------
o Now lets start analysing our first model
------------------------------------------------------
(We don't care for normals because they can be autogenerated in most 3D tools.)
We leave the "LitE" button as it is (little endian data).

The face indices are very easy to find since parts of the indices block look like a scrambled alphabet.

If we scroll through hw_bu_000.bm in a hex editor we'll find this:

To find the correct ending of the list is a very important step.
Here it's rather easy (0x4845) because it's bound by a string size (09 000000)
and the corresponding string.

mapping of the tutorial example:
• 0x56 ..0x15C1 vertices
• 0x15C2..0x2B2D normals
• 0x2B2E length DWord and string "Texture"
• 0x2B39..0x3980 uvs, length: 0xE48
Rest of the tutorial to be found in the zip. Because you need to start hex2obj.exe
to get full use of the tutorial.

---------- >>> version 0.24c <<< ----------
hex2obj_0.24c.zip
- uv preview
- allows face indices > 65535 (for non-tristripped meshes)

---------- version 0.24d, autogenerated FIs ----------
viewtopic.php?f=29&t=10894&p=119417&hil ... ed#p119417
---------- version 0.24e, unfinished ----------
viewtopic.php?f=29&t=10894&p=142434&hil ... al#p142434

Having up to 3 or 4 downloads per day atm I feel obliged to tell you guys once again:
hex2obj is NOT a one-click-solver tool - it was designed to help people to help themselves.

If you don't have a basic understanding of 3D model formats you surely will get better results using a 3D ripper (for PC).

With this said I think it's time to make a check list for noobs concerning the analyzing of 3D models:

Code: Select all

``````1) is the data compressed?
Try offzip if your unsure (works for zip compressed data only) or search for a decompressing tool.
(There's also Quickbms scripts for many archive formats.)

2) is it model data, texture data or animation data?
Use TextureFinder for textures.
Some 3D models may contain both: model data and texture(s)

3) Before using hex2obj on a difficult format try out some of the dozens of simple samples on Xentax.

4) Then try out to get a point cloud  from your specific model as a first step.

Look for continuous data sections with a size of 1 kB at least (0x400).

Yeah, what the heck does that mean: "continuous data"?

Hard to explain - in fact you'll need experience for that.
If there's gaps with more than 10 zero bytes for example I'd say it's not continuous data.

While float data is easy to recognize (with some experience) it's
rather hard to find vertex data in short/Word format.``````
The v0.21 H2O format changed slightly. So you can't use it with elder versions of the hex2obj.
hex2obj (v0.21) can read elder H2O files and will convert them to v0.21 when saving them.

(Be aware that there are various possible combinations of little/big endian, seq/VB, WORD/DW, noStrips/Strips and half floats/or not.
With any new version dozens of functions to be updated so there might be another bug.)

---------- These outdated versions should be used as a fallback only! ------------
v. 0.22
hex2obj_0.22.zip

v 0.23b, exe only
Introduces "multi mesh" creation and the table button.
-----------------------------------------------------------------

known bugs: v0.1: TriStrips starting on even addresses only
v0.2: wrong HF-uvs displayed in list box; ok with 0.2x
v0.22: for rare cases when there is a '.' in a folder name:
v0.22c: TriStrips, superfluous degenerated faces
v0.23d: MMesh feature accidently disabled
v0.23d-f intermediate versions with bug, don't use!
v0.24a uvs preview (float/VB)

reported bugs:
v0.21: wrong clipping for HF uvs
........ kao issue
v0.24: libgcc_s_dw2-1.dll missing on W7, W8 (due to CodeBlocks/MinGW update)

todo list

HTH,
shak-otay
Last edited by shakotay2 on Thu Jul 26, 2018 1:43 am, edited 55 times in total.
Bigchillghost, Reverse Engineering a Game Model: viewtopic.php?f=29&t=17889
extracting simple models: viewtopic.php?f=29&t=10894
Make_H2O-ForzaHor3-jm9.zip
"You quoted the whole thing, what a mess."

shakotay2
MEGAVETERAN
Posts: 2605
Joined: Fri Apr 20, 2012 9:24 am
Location: Nexus, searching for Jim Kirk
Has thanked: 638 times
Been thanked: 1332 times

### Re: Extracting simple models

FAQs

Q: how to find the start addresses of vertices and face indices (FIs)?
A: as for FIs, search for a "scrambled alphabet" but in general it's like this:
I don't know where vertices end and where faces start. The only way is make an assumption, check it. If its wrong, make another one, and check again. With time and experience, you will usually be able to make it right from the first try. (daemon1)
Q: when I press the 'mesh' button it gives me an exception.
A: A problem with the external mesh_viewer app. It expects a proper obj.
But hex2obj will not always provide a proper one.
It's some kind of trial 'n error tool, you know?

(With some knowledge you could have a look at test.obj to see what's wrong.)

Q: when I press the 'mesh' button nothing happens.
A: have a look at the output of the console window. If it's like this

Usage: D:\CodeBlocks\Hex2obj\mesh_viewer.exe [-grab] infile..."

Q: how to calculate the vertex count of a (sub) mesh?
A: hex2obj doesn't need to search for the vertex count because
it's using the max face index value as vertex count.

That's the fundamental idea behind the project.

Q: what about the '99' UVpos in VB mode?
A: means: no UVs used. So with a VBsize of 12 and HF_no
test.obj should contain same data as in seq mode.

Q: why didn't you write this tool 5 years ago?
A: I r-e-a-l-l-y don't know. (Would have spared me tons of lifetime. )

Q: why the f... is Hex2obj's GUI messed up with Win7?

A: happens, when you use a bigger font size in the Win7 GUI.
Since this seems to be rare I won't upload a separate exe version.
(Feel free to request one if required.)

DIZZEL
viewtopic.php?f=10&t=12708&p=104615&hil ... bj#p104615

more detailed hints on use of hex2obj:

another short tutorial:
viewtopic.php?f=16&t=12429&p=102267&hil ... ed#p102267

Kinetec Star Wars
viewtopic.php?f=16&t=11436

LittleBigPlanet 3 .mol
viewtopic.php?f=16&t=12646&p=104214&hil ... bj#p104214

LithTech model
viewtopic.php?f=16&t=12578&p=103497&hil ... bj#p103497

submeshes of simple models
Flintstones Bedrock Bowling .AMW
viewtopic.php?f=16&t=12521&p=103091#p103091

preview uvs feature:
Elite Dangerous
viewtopic.php?f=10&t=11752&hilit=hex2obj&start=15

absolute/relative face indices:
Ride (by Milestone) .BIN format
viewtopic.php?f=16&t=12791&p=105798&hil ... bj#p105798

hex2obj/Make_H2O
Wipeout HD/Fury
viewtopic.php?f=16&t=12906&p=106658&hil ... bj#p106658

Shining Resonance .MLX
viewtopic.php?f=16&t=12392&hilit=hex2obj&start=30

NeverDead viewtopic.php?f=16&t=10960, post Thu Nov 14, 2013 12:06 pm
young justice legacy viewtopic.php?f=16&t=11014, post Fri Dec 06, 2013 9:55 am
MDK 2 viewtopic.php?f=16&t=11036&p=90803#p90803, post Thu Dec 12, 2013 12:51 pm
Last edited by shakotay2 on Fri Jan 20, 2017 2:41 pm, edited 18 times in total.
Bigchillghost, Reverse Engineering a Game Model: viewtopic.php?f=29&t=17889
extracting simple models: viewtopic.php?f=29&t=10894
Make_H2O-ForzaHor3-jm9.zip
"You quoted the whole thing, what a mess."

shakotay2
MEGAVETERAN
Posts: 2605
Joined: Fri Apr 20, 2012 9:24 am
Location: Nexus, searching for Jim Kirk
Has thanked: 638 times
Been thanked: 1332 times

### Re: Extracting simple models

A model from this thread: viewtopic.php?f=10&t=8368&p=89143#p89143

updated exe (big endian untested) in start post.
Bigchillghost, Reverse Engineering a Game Model: viewtopic.php?f=29&t=17889
extracting simple models: viewtopic.php?f=29&t=10894
Make_H2O-ForzaHor3-jm9.zip
"You quoted the whole thing, what a mess."

shekofte
mega-veteran
Posts: 218
Joined: Sun Jan 18, 2009 1:45 pm
Location: Sagittarius
Has thanked: 235 times
Been thanked: 12 times
Contact:

### Re: Extracting simple models

nice !
in near future i hope you extend these tutorials to more advanced levels of a typical modern 3d model like normals , texture coordinates ...
ILLUSORY VIDEO GAMES ARE MORE IMPORTANT THAN PRECIOUS INTERNATIONAL FOOTBALLs

shakotay2
MEGAVETERAN
Posts: 2605
Joined: Fri Apr 20, 2012 9:24 am
Location: Nexus, searching for Jim Kirk
Has thanked: 638 times
Been thanked: 1332 times

### Re: Extracting simple models

Normals are of subordinate meaning imho because you can autocreate them in nearly every 3D tool - as I wrote.

texture coords (uvs) are covered.

But your right: the models in this thread viewtopic.php?f=16&t=10841&p=89008#p89008 use a vertex block where the uv data is supposed to be half floats.

Code: Select all

``````Offset  0  1  2  3  4  5  6  7   8  9  A  B  C  D  E  F
0A80   BF 54 71 82 42 87 D5 A8  3F B1 95 57 25 10 83 12
0A90   80 49 18 65 02 00 00 00  FF FF 00 00 00 00 00 00``````
Where the first 12 bytes are the x,y,z position and the following 8 bytes might be 4 half floats
vtx1,vty1, vtx2,vty2?

edit: interpreting bytes 12..15 as 2 words, first divided by 32768.0, 2nd by 65536.0 you'll get
vt 0.289551 0.511993
vt 0.501404 0.542847
vt 0.073792 0.800491
vt 0.251343 0.551270
vt 0.289551 0.511993
vt 0.438599 0.538651
vt 0.838684 0.787979
...
From the values that looks pretty nice but isn't. This is some garbage but not an uv map.
So finding the the uvs might drive you carzy.

Automated analysation will always be limited.
This is a tool for noobs to reduce the threshold.
In the end there's no other choice than using your brain on analyzing of more complex models.
Last edited by shakotay2 on Fri Nov 01, 2013 1:20 pm, edited 2 times in total.
Bigchillghost, Reverse Engineering a Game Model: viewtopic.php?f=29&t=17889
extracting simple models: viewtopic.php?f=29&t=10894
Make_H2O-ForzaHor3-jm9.zip
"You quoted the whole thing, what a mess."

luxox18
mega-veteran
Posts: 176
Joined: Fri Jul 29, 2011 9:18 pm
Has thanked: 54 times
Been thanked: 41 times

### Re: Extracting simple models

Is possible in the future load the entire model and not only submeshes? (for example, the model from dante's inferno)

Bastien
Posts: 69
Joined: Sun Apr 15, 2012 1:08 am
Has thanked: 26 times
Been thanked: 12 times

### Re: Extracting simple models

Sounds like a pretty cool tool to also test unknown formats!
will test it later
cheers

shakotay2
MEGAVETERAN
Posts: 2605
Joined: Fri Apr 20, 2012 9:24 am
Location: Nexus, searching for Jim Kirk
Has thanked: 638 times
Been thanked: 1332 times

### Re: Extracting simple models

luxox18 wrote:Is possible in the future load the entire model and not only submeshes? (for example, the model from dante's inferno)
It's planned to introduce a param's file where to store the data of submeshes (vertex start addresses, vertex counts). But this will not release you from searching for the submeshes manually before.

Building the entire model from this param file's data should be possible.

edit: version 0.1 (view start post) can handle half floats and triangle strips.
Load/save of a H2O (param) file is possible.
Composing of submeshes to an entire model not implemented because of
possible "65535 indices border" crossing (DWORD required).

Although it's not really difficult, it has low priority for me.
This is the reason: atm you'll have to search manually for every single submesh (SM) of the model.
Often there are patterns by which you can identify the start addresses of the SMs.

But you need to know these patterns which are different with every game.
(And for some models there aren't such patterns.)
I can't think of a feature how to get them automatically.

Once you know the pattern you can find the SMs by using your hex editors search function.

You can search for 00 00 01 00 02 00 (00 00 00 01 00 02 for big endian) as an identifier for the start of WORD face indices lists but that does apply to few games only.
Last edited by shakotay2 on Sat Nov 16, 2013 7:25 am, edited 1 time in total.
Bigchillghost, Reverse Engineering a Game Model: viewtopic.php?f=29&t=17889
extracting simple models: viewtopic.php?f=29&t=10894
Make_H2O-ForzaHor3-jm9.zip
"You quoted the whole thing, what a mess."

shakotay2
MEGAVETERAN
Posts: 2605
Joined: Fri Apr 20, 2012 9:24 am
Location: Nexus, searching for Jim Kirk
Has thanked: 638 times
Been thanked: 1332 times

### Re: Extracting simple models

Handling triangle strips with version 0.1
the model and the pnzgren.h2o (param) file.

You'll have to toggle the 'std' button to 'FFFF' (separator) bevor you click the 'mesh' button.

Getting further submeshes goes as follows:
well, we find another scrambled alphabet at 0x12620. The list ends at 0x1308D.
So there are 0xA6E bytes or 2670/2= 1335 face indices.

Entering these values and clicking 'go1' reveals a vertex count of 755 at the lower left listboxes' end.

The start address of the vertices block is easily to be found: whole vertex list is 755*VBsize= 18120 dec.
Subtracting 0x46C8 from 0x12260 gives 0xDF58 as the vertices start address.
File SaveAs mesh et voila: (with normals drawn)

(the status of the 'std/FFFF' button is not saved into a H2O file so far)

edit: for the vertices start of the body mesh you'll have to stick to 0x1BB88.
(There's some 'blurring' between calculated (2032) and factual (2044) vertices count, hehehe.)
Last edited by shakotay2 on Fri Nov 01, 2013 11:08 am, edited 1 time in total.
Bigchillghost, Reverse Engineering a Game Model: viewtopic.php?f=29&t=17889
extracting simple models: viewtopic.php?f=29&t=10894
Make_H2O-ForzaHor3-jm9.zip
"You quoted the whole thing, what a mess."

shakotay2
MEGAVETERAN
Posts: 2605
Joined: Fri Apr 20, 2012 9:24 am
Location: Nexus, searching for Jim Kirk
Has thanked: 638 times
Been thanked: 1332 times

### Re: Extracting simple models

version 0.2 is out. Can handle vertices/UV data as WORDs (unsigned 16 bit values).

(As always I hope I didn't mess up too many things.)

The use of the 'cut'-button might be required for "Word_all" models:

("Word_all" simply means that there are 3 WORDs for the vertex pos and 2 WORDs for the UV data.)

edit:
with the fixed version 0.2 I got a rage bmodel:

but .bmodel is not always triangle strips as you can see:

edit2: ships always being my preferred models. Here's locr00_spielerschiff_s_0.nvx2 from Drakensang, ROT:

UV coords as words (looks like a divisor of 8000 being rather better than 65536).

edit3: space-ships:
viewtopic.php?f=16&t=11436
Last edited by shakotay2 on Mon Apr 21, 2014 7:40 am, edited 7 times in total.
Bigchillghost, Reverse Engineering a Game Model: viewtopic.php?f=29&t=17889
extracting simple models: viewtopic.php?f=29&t=10894
Make_H2O-ForzaHor3-jm9.zip
"You quoted the whole thing, what a mess."

shakotay2
MEGAVETERAN
Posts: 2605
Joined: Fri Apr 20, 2012 9:24 am
Location: Nexus, searching for Jim Kirk
Has thanked: 638 times
Been thanked: 1332 times

### Re: Extracting simple models

Version 0.2x introducing "mixed mode" (for advanced users only!)

From the tut you know there are two main ways how the model data is organized:
"sequential", that means there are all vertices (v) then all normals (vn) to follow
then all uvs (vt). (May also be: all v, all vt, all vn.)

Another possibility is the "blocked" mode.
Means you have a Vertex Block with a size of let's say 32 bytes.
First block contains first vertex, first normal, first uv.
2nd block contains 2nd vertex, 2nd nomal, 2nd uv.
And so on.
(That's an example only! There may be other data in the blocks, such as DWords.)

Now we introduce the "mixed" mode where vertices and uvs don't share the same block.
---------------------------------

You may have a vertex block with a size of 16 bytes (3 floats vertex, 1 float unknown),
then the 2nd block and so on.

After the last vertex block the uv blocks start.
May have a size of 12 bytes (2 floats UV, 1 unknown DWORD)
Then the 2nd uv block follows and so on.

I call it "mixed" mode because you can look upon it as a blocked AND as a sequential mode, too.
"Sequential" because there are all vertices, then all uvs.
But vertices and uvs are in blocks, too. Not with each other but together with other data.

As an example you might take the model from this thread:
viewtopic.php?f=16&t=11056&p=91650#p91650
AcheronGolem_weapon.JPG
----------

NOT saved into the H2O file so far.
You do not have the required permissions to view the files attached to this post.
Last edited by shakotay2 on Mon Jan 20, 2014 1:52 pm, edited 1 time in total.
Bigchillghost, Reverse Engineering a Game Model: viewtopic.php?f=29&t=17889
extracting simple models: viewtopic.php?f=29&t=10894
Make_H2O-ForzaHor3-jm9.zip
"You quoted the whole thing, what a mess."

luxox18
mega-veteran
Posts: 176
Joined: Fri Jul 29, 2011 9:18 pm
Has thanked: 54 times
Been thanked: 41 times

### Re: Extracting simple models

With hex2obj 0.2x now I can obtain UV from Dante's inferno models

thanks shakotay2

Doronetty
Posts: 49
Joined: Thu Jun 03, 2010 11:05 am
Has thanked: 24 times
Been thanked: 3 times

### Re: Extracting simple models

Acewell
VIP member
Posts: 1186
Joined: Wed Nov 05, 2008 12:16 pm
Has thanked: 2330 times
Been thanked: 686 times

### Re: Extracting simple models

I have the file and could upload it as last resort, but this really should be added to the Tools blog.

shakotay2
MEGAVETERAN
Posts: 2605
Joined: Fri Apr 20, 2012 9:24 am
Location: Nexus, searching for Jim Kirk
Has thanked: 638 times
Been thanked: 1332 times

### Re: Extracting simple models

thx - updated start post

--------------------------
(This is a text backup only for maybe future quotations.)

Workaround for formats which use shorts for the vertices and half floats for the uvs:

save the mesh in two ways, first as ShortAll then as HF_UV (or HF_all).
In the first obj file do a replacement of 'vt' by '#vt'.
In the (HF_UV) obj file do a replacement of 'v ' by '#v '. (<<< Don't forget the space here.)

Then copy all vts (without the face indices) into the (ShortAll) obj.

(Yep, this is ugly but should work.)
Last edited by shakotay2 on Sun Apr 24, 2016 1:00 pm, edited 3 times in total.
Bigchillghost, Reverse Engineering a Game Model: viewtopic.php?f=29&t=17889
extracting simple models: viewtopic.php?f=29&t=10894
Make_H2O-ForzaHor3-jm9.zip
"You quoted the whole thing, what a mess."