It is currently Fri Apr 28, 2017 5:21 am

 All times are UTC + 1 hour

 Page 1 of 18 [ 259 posts ] Go to page 1, 2, 3, 4, 5 ... 18  Next
 Print view Previous topic | Next topic
Author Message
 Post subject: Extracting simple modelsPosted: Tue Oct 22, 2013 3:06 pm
 M-M-M-Monster veteran

Joined: Fri Apr 20, 2012 9:24 am
Posts: 1995
Has thanked: 358 times
Have thanks: 991 times

```

```

You can make the ads go away by registering

o noobs' hex to obj conversion (poor knowledge required)

It should allow u to extract 3D models with simple formats.
Means such games like Mafia2 or SleepingDogs not to be supported.

- preliminary -

Ok, the truth is - there are two things you should understand:

the hexadecimal numeral system (base 16) using the "numbers" 0..9, A..F
So 0x123 is 291 decimal.
(To avoid confusion with other numbering systems the hex numbers are marked with 0x or an 'h'.)

Secondly you should be able to use a hex editor.

If you don't have these two skills leave now, and come back as soon as you gained them.

(If you don't have a clue which hex editor to use: TinyHexer or HexEdit by A.W. Phillips should do the trick.)
--------------------------------------

As I said you don't need to learn "basics" but you'll have to understand
"the structures" to use hex2obj.

o So here is a short tutorial on it's use.

You don't need to know the technical terms - I just use them habitual.

Simple 3D models consist of vertices and faces.

hex2obj needs the following infos to perform it's task:
the startaddress of the vertices and the one of the faceindices.

(Some experience is required to find them but it's not too hard.)

Secondly we need the vertices count and the faceindices count.

I will explain this with a model example soon but now for the structures -

the two most common are:
a) "sequential"
Code:
v x,y,z
v x,y,z
v x,y,z
v x,y,z
...
vn x,y,z
vn x,y,z
vn x,y,z
vn x,y,z
...
vt x,y
vt x,y
vt x,y
vt x,y
...
(v: vertex position, vn: normals, vt: uv)
With a sequential model you only need to provide the above mentioned counts.

b) "blocked"
Code:
v  x,y,z
vn x,y,z
vt x,y

v  x,y,z
vn x,y,z
vt x,y
...
With a blocked model you need to chose the vertex block size (FVF size), for example 32.
(And 'yes': the order might be v, vt, vn for example)

If you don't understand anything you'll have to do a trial 'n error.

Or you calculate the FVF size: each coordinate (x, y or z) consists of 4 bytes.
So the above two blocks have a blocksize of 32 (8x4) each.

It might be a little bit more difficult if there are DWORDs contained in the blocks.
But anyway the most common blocksizes are 32, 40, 52. Sometimes 24, 44, 48 whatever.

o the v,vn,vt patterns to search for

well, to identify what is v, vn or vt is the tricky part. This is for little endian
(for big endian the patterns are in reverse order of course).

v: xx xx xx nn with nn= 42..44 (these are rough values only, can also be 3F, 40)
vn: value of each component 0.0 .. 1.0 (1.0 = 00 00 80 3F)
square sum of components is 1.0 (autocalculated by hex2obj)
vt: value of each component 0.0 .. 1.0

It's important to know that half floats use two bytes only.

------------------------------------------------------
o Now lets start analysing our first model
------------------------------------------------------
(We don't care for normals because they can be autogenerated in most 3D tools.)
We leave the "LitE" button as it is (little endian data).

The face indices are very easy to find since parts of the indices block look like a scrambled alphabet.

If we scroll through hw_bu_000.bm in a hex editor we'll find this:

To find the correct ending of the list is a very important step.
Here it's rather easy (0x4845) because it's bound by a string size (09 000000)
and the corresponding string.

mapping of the tutorial example:
• 0x56 ..0x15C1 vertices
• 0x15C2..0x2B2D normals
• 0x2B2E length DWord and string "Texture"
• 0x2B39..0x3980 uvs, length: 0xE48
Rest of the tutorial to be found in the zip. Because you need to start hex2obj.exe
to get full use of the tutorial.

---------- >>> version 0.24c <<< ----------
hex2obj_0.24c.zip
- uv preview
- allows face indices > 65535 (for non-tristripped meshes)

Having up to 3 or 4 downloads per day atm I feel obliged to tell you guys once again:
hex2obj is NOT a one-click-solver tool - it was designed to help people to help themselves.

If you don't have a basic understanding of 3D model formats you surely will get better results using a 3D ripper (for PC).

With this said I think it's time to make a check list for noobs concerning the analyzing of 3D models:
Code:
1) is the data compressed?
Try offzip if your unsure (works for zip compressed data only) or search for a decompressing tool.
(There's also Quickbms scripts for many archive formats.)

2) is it model data, texture data or animation data?
Use TextureFinder for textures.
Some 3D models may contain both: model data and texture(s)

3) Before using hex2obj on a difficult format try out some of the dozens of simple samples on Xentax.

4) Then try out to get a point cloud  from your specific model as a first step.

Look for continuous data sections with a size of 1 kB at least (0x400).

Yeah, what the heck does that mean: "continuous data"?

Hard to explain - in fact you'll need experience for that.
If there's gaps with more than 10 zero bytes for example I'd say it's not continuous data.

While float data is easy to recognize (with some experience) it's
rather hard to find vertex data in short/Word format.

The v0.21 H2O format changed slightly. So you can't use it with elder versions of the hex2obj.
hex2obj (v0.21) can read elder H2O files and will convert them to v0.21 when saving them.

(Be aware that there are various possible combinations of little/big endian, seq/VB, WORD/DW, noStrips/Strips and half floats/or not.
With any new version dozens of functions to be updated so there might be another bug.)

---------- These outdated versions should be used as a fallback only! ------------
v. 0.22
hex2obj_0.22.zip

v 0.23b, exe only
Introduces "multi mesh" creation and the table button.
-----------------------------------------------------------------

known bugs: v0.1: TriStrips starting on even addresses only
v0.2: wrong HF-uvs displayed in list box; ok with 0.2x
v0.22: for rare cases when there is a '.' in a folder name:
v0.22c: TriStrips, superfluous degenerated faces
v0.23d: MMesh feature accidently disabled
v0.23d-f intermediate versions with bug, don't use!
v0.24a uvs preview (float/VB)

reported bugs:
v0.21: wrong clipping for HF uvs
........ kao issue
v0.24: libgcc_s_dw2-1.dll missing on W7, W8 (due to CodeBlocks/MinGW update)

todo list

HTH,
shak-otay

_________________
Extracting simple models: http://forum.xentax.com/viewtopic.php?f=29&t=10894
Make_H2O-ForzaHor3-jm9.zip

"We are Microsoft. You will be assimilated. Resistance is Futile."

Last edited by shakotay2 on Tue Mar 28, 2017 8:07 pm, edited 53 times in total.

Top

 Post subject: Re: Extracting simple modelsPosted: Tue Oct 22, 2013 3:09 pm
 M-M-M-Monster veteran

Joined: Fri Apr 20, 2012 9:24 am
Posts: 1995
Has thanked: 358 times
Have thanks: 991 times
FAQs

Q: how to find the start addresses of vertices and face indices (FIs)?
A: as for FIs, search for a "scrambled alphabet" but in general it's like this:
Quote:
I don't know where vertices end and where faces start. The only way is make an assumption, check it. If its wrong, make another one, and check again. With time and experience, you will usually be able to make it right from the first try. (daemon1)

Q: when I press the 'mesh' button it gives me an exception.
A: A problem with the external mesh_viewer app. It expects a proper obj.
But hex2obj will not always provide a proper one.
It's some kind of trial 'n error tool, you know?

(With some knowledge you could have a look at test.obj to see what's wrong.)

Q: when I press the 'mesh' button nothing happens.
A: have a look at the output of the console window. If it's like this

Usage: D:\CodeBlocks\Hex2obj\mesh_viewer.exe [-grab] infile..."

Q: how to calculate the vertex count of a (sub) mesh?
A: hex2obj doesn't need to search for the vertex count because
it's using the max face index value as vertex count.

That's the fundamental idea behind the project.

Q: what about the '99' UVpos in VB mode?
A: means: no UVs used. So with a VBsize of 12 and HF_no
test.obj should contain same data as in seq mode.

Q: why didn't you write this tool 5 years ago?
A: I r-e-a-l-l-y don't know. (Would have spared me tons of lifetime. )

Q: why the f... is Hex2obj's GUI messed up with Win7?

A: happens, when you use a bigger font size in the Win7 GUI.
Since this seems to be rare I won't upload a separate exe version.
(Feel free to request one if required.)

DIZZEL
viewtopic.php?f=10&t=12708&p=104615&hilit=hex2obj#p104615

more detailed hints on use of hex2obj:

another short tutorial:
viewtopic.php?f=16&t=12429&p=102267&hilit=scrambled#p102267

Kinetec Star Wars
viewtopic.php?f=16&t=11436

LittleBigPlanet 3 .mol
viewtopic.php?f=16&t=12646&p=104214&hilit=hex2obj#p104214

LithTech model
viewtopic.php?f=16&t=12578&p=103497&hilit=hex2obj#p103497

submeshes of simple models
Flintstones Bedrock Bowling .AMW
viewtopic.php?f=16&t=12521&p=103091#p103091

preview uvs feature:
Elite Dangerous
viewtopic.php?f=10&t=11752&hilit=hex2obj&start=15

absolute/relative face indices:
Ride (by Milestone) .BIN format
viewtopic.php?f=16&t=12791&p=105798&hilit=hex2obj#p105798

hex2obj/Make_H2O
Wipeout HD/Fury
viewtopic.php?f=16&t=12906&p=106658&hilit=+hex2obj#p106658

Shining Resonance .MLX
viewtopic.php?f=16&t=12392&hilit=hex2obj&start=30

NeverDead http://forum.xentax.com/viewtopic.php?f=16&t=10960, post Thu Nov 14, 2013 12:06 pm
young justice legacy http://forum.xentax.com/viewtopic.php?f=16&t=11014, post Fri Dec 06, 2013 9:55 am
MDK 2 http://forum.xentax.com/viewtopic.php?f=16&t=11036&p=90803#p90803, post Thu Dec 12, 2013 12:51 pm

_________________
Extracting simple models: http://forum.xentax.com/viewtopic.php?f=29&t=10894
Make_H2O-ForzaHor3-jm9.zip

"We are Microsoft. You will be assimilated. Resistance is Futile."

Last edited by shakotay2 on Fri Jan 20, 2017 2:41 pm, edited 18 times in total.

Top

 Post subject: Re: Extracting simple modelsPosted: Wed Oct 23, 2013 2:35 pm
 M-M-M-Monster veteran

Joined: Fri Apr 20, 2012 9:24 am
Posts: 1995
Has thanked: 358 times
Have thanks: 991 times
A model from this thread: viewtopic.php?f=10&t=8368&p=89143#p89143

updated exe (big endian untested) in start post.

_________________
Extracting simple models: http://forum.xentax.com/viewtopic.php?f=29&t=10894
Make_H2O-ForzaHor3-jm9.zip

"We are Microsoft. You will be assimilated. Resistance is Futile."

Top

 Post subject: Re: Extracting simple modelsPosted: Wed Oct 23, 2013 3:05 pm
 mega-veteran

Joined: Sun Jan 18, 2009 1:45 pm
Posts: 205
Location: Sagittarius
Has thanked: 198 times
Have thanks: 12 times
nice !
in near future i hope you extend these tutorials to more advanced levels of a typical modern 3d model like normals , texture coordinates ...

_________________
ILLUSORY VIDEO GAMES ARE MORE IMPORTANT THAN PRECIOUS INTERNATIONAL FOOTBALLs

Top

 Post subject: Re: Extracting simple modelsPosted: Wed Oct 23, 2013 3:10 pm
 M-M-M-Monster veteran

Joined: Fri Apr 20, 2012 9:24 am
Posts: 1995
Has thanked: 358 times
Have thanks: 991 times
Normals are of subordinate meaning imho because you can autocreate them in nearly every 3D tool - as I wrote.

texture coords (uvs) are covered.

But your right: the models in this thread viewtopic.php?f=16&t=10841&p=89008#p89008 use a vertex block where the uv data is supposed to be half floats.

Code:
Offset  0  1  2  3  4  5  6  7   8  9  A  B  C  D  E  F
0A80   BF 54 71 82 42 87 D5 A8  3F B1 95 57 25 10 83 12
0A90   80 49 18 65 02 00 00 00  FF FF 00 00 00 00 00 00

Where the first 12 bytes are the x,y,z position and the following 8 bytes might be 4 half floats
vtx1,vty1, vtx2,vty2?

edit: interpreting bytes 12..15 as 2 words, first divided by 32768.0, 2nd by 65536.0 you'll get
vt 0.289551 0.511993
vt 0.501404 0.542847
vt 0.073792 0.800491
vt 0.251343 0.551270
vt 0.289551 0.511993
vt 0.438599 0.538651
vt 0.838684 0.787979
...
From the values that looks pretty nice but isn't. This is some garbage but not an uv map.
So finding the the uvs might drive you carzy.

Automated analysation will always be limited.
This is a tool for noobs to reduce the threshold.
In the end there's no other choice than using your brain on analyzing of more complex models.

_________________
Extracting simple models: http://forum.xentax.com/viewtopic.php?f=29&t=10894
Make_H2O-ForzaHor3-jm9.zip

"We are Microsoft. You will be assimilated. Resistance is Futile."

Last edited by shakotay2 on Fri Nov 01, 2013 1:20 pm, edited 2 times in total.

Top

 Post subject: Re: Extracting simple modelsPosted: Thu Oct 24, 2013 2:34 am
 mega-veteran

Joined: Fri Jul 29, 2011 9:18 pm
Posts: 175
Has thanked: 52 times
Have thanks: 38 times
Is possible in the future load the entire model and not only submeshes? (for example, the model from dante's inferno)

Top

 Post subject: Re: Extracting simple modelsPosted: Thu Oct 24, 2013 2:41 am

Joined: Sun Apr 15, 2012 1:08 am
Posts: 69
Has thanked: 26 times
Have thanks: 12 times
Sounds like a pretty cool tool to also test unknown formats!
will test it later
cheers

Top

 Post subject: Re: Extracting simple modelsPosted: Thu Oct 24, 2013 9:39 am
 M-M-M-Monster veteran

Joined: Fri Apr 20, 2012 9:24 am
Posts: 1995
Has thanked: 358 times
Have thanks: 991 times
luxox18 wrote:
Is possible in the future load the entire model and not only submeshes? (for example, the model from dante's inferno)
It's planned to introduce a param's file where to store the data of submeshes (vertex start addresses, vertex counts). But this will not release you from searching for the submeshes manually before.

Building the entire model from this param file's data should be possible.

edit: version 0.1 (view start post) can handle half floats and triangle strips.
Load/save of a H2O (param) file is possible.
Composing of submeshes to an entire model not implemented because of
possible "65535 indices border" crossing (DWORD required).

Although it's not really difficult, it has low priority for me.
This is the reason: atm you'll have to search manually for every single submesh (SM) of the model.
Often there are patterns by which you can identify the start addresses of the SMs.

But you need to know these patterns which are different with every game.
(And for some models there aren't such patterns.)
I can't think of a feature how to get them automatically.

Once you know the pattern you can find the SMs by using your hex editors search function.

You can search for 00 00 01 00 02 00 (00 00 00 01 00 02 for big endian) as an identifier for the start of WORD face indices lists but that does apply to few games only.

_________________
Extracting simple models: http://forum.xentax.com/viewtopic.php?f=29&t=10894
Make_H2O-ForzaHor3-jm9.zip

"We are Microsoft. You will be assimilated. Resistance is Futile."

Last edited by shakotay2 on Sat Nov 16, 2013 7:25 am, edited 1 time in total.

Top

 Post subject: Re: Extracting simple modelsPosted: Tue Oct 29, 2013 12:28 am
 M-M-M-Monster veteran

Joined: Fri Apr 20, 2012 9:24 am
Posts: 1995
Has thanked: 358 times
Have thanks: 991 times
Handling triangle strips with version 0.1
the model and the pnzgren.h2o (param) file.

You'll have to toggle the 'std' button to 'FFFF' (separator) bevor you click the 'mesh' button.

Getting further submeshes goes as follows:
well, we find another scrambled alphabet at 0x12620. The list ends at 0x1308D.
So there are 0xA6E bytes or 2670/2= 1335 face indices.

Entering these values and clicking 'go1' reveals a vertex count of 755 at the lower left listboxes' end.

The start address of the vertices block is easily to be found: whole vertex list is 755*VBsize= 18120 dec.
Subtracting 0x46C8 from 0x12260 gives 0xDF58 as the vertices start address.
File SaveAs mesh et voila: (with normals drawn)

(the status of the 'std/FFFF' button is not saved into a H2O file so far)

edit: for the vertices start of the body mesh you'll have to stick to 0x1BB88.
(There's some 'blurring' between calculated (2032) and factual (2044) vertices count, hehehe.)

_________________
Extracting simple models: http://forum.xentax.com/viewtopic.php?f=29&t=10894
Make_H2O-ForzaHor3-jm9.zip

"We are Microsoft. You will be assimilated. Resistance is Futile."

Last edited by shakotay2 on Fri Nov 01, 2013 11:08 am, edited 1 time in total.

Top

 Post subject: Re: Extracting simple modelsPosted: Tue Nov 26, 2013 5:42 pm
 M-M-M-Monster veteran

Joined: Fri Apr 20, 2012 9:24 am
Posts: 1995
Has thanked: 358 times
Have thanks: 991 times
version 0.2 is out. Can handle vertices/UV data as WORDs (unsigned 16 bit values).

(As always I hope I didn't mess up too many things.)

The use of the 'cut'-button might be required for "Word_all" models:

("Word_all" simply means that there are 3 WORDs for the vertex pos and 2 WORDs for the UV data.)

edit:
with the fixed version 0.2 I got a rage bmodel:

but .bmodel is not always triangle strips as you can see:

edit2: ships always being my preferred models. Here's locr00_spielerschiff_s_0.nvx2 from Drakensang, ROT:

UV coords as words (looks like a divisor of 8000 being rather better than 65536).

edit3: space-ships:
viewtopic.php?f=16&t=11436

_________________
Extracting simple models: http://forum.xentax.com/viewtopic.php?f=29&t=10894
Make_H2O-ForzaHor3-jm9.zip

"We are Microsoft. You will be assimilated. Resistance is Futile."

Last edited by shakotay2 on Mon Apr 21, 2014 7:40 am, edited 7 times in total.

Top

 Post subject: Re: Extracting simple modelsPosted: Wed Nov 27, 2013 9:38 am
 M-M-M-Monster veteran

Joined: Fri Apr 20, 2012 9:24 am
Posts: 1995
Has thanked: 358 times
Have thanks: 991 times
Version 0.2x introducing "mixed mode" (for advanced users only!)

From the tut you know there are two main ways how the model data is organized:
"sequential", that means there are all vertices (v) then all normals (vn) to follow
then all uvs (vt). (May also be: all v, all vt, all vn.)

Another possibility is the "blocked" mode.
Means you have a Vertex Block with a size of let's say 32 bytes.
First block contains first vertex, first normal, first uv.
2nd block contains 2nd vertex, 2nd nomal, 2nd uv.
And so on.
(That's an example only! There may be other data in the blocks, such as DWords.)

Now we introduce the "mixed" mode where vertices and uvs don't share the same block.
---------------------------------

You may have a vertex block with a size of 16 bytes (3 floats vertex, 1 float unknown),
then the 2nd block and so on.

After the last vertex block the uv blocks start.
May have a size of 12 bytes (2 floats UV, 1 unknown DWORD)
Then the 2nd uv block follows and so on.

I call it "mixed" mode because you can look upon it as a blocked AND as a sequential mode, too.
"Sequential" because there are all vertices, then all uvs.
But vertices and uvs are in blocks, too. Not with each other but together with other data.

As an example you might take the model from this thread:
viewtopic.php?f=16&t=11056&p=91650#p91650

----------

NOT saved into the H2O file so far.

Attachments:

You do not have the required permissions to view the files attached to this post. Register to gain access.

_________________
Extracting simple models: http://forum.xentax.com/viewtopic.php?f=29&t=10894
Make_H2O-ForzaHor3-jm9.zip

"We are Microsoft. You will be assimilated. Resistance is Futile."

Last edited by shakotay2 on Mon Jan 20, 2014 1:52 pm, edited 1 time in total.

Top

 Post subject: Re: Extracting simple modelsPosted: Wed Dec 11, 2013 2:28 am
 mega-veteran

Joined: Fri Jul 29, 2011 9:18 pm
Posts: 175
Has thanked: 52 times
Have thanks: 38 times
With hex2obj 0.2x now I can obtain UV from Dante's inferno models

thanks shakotay2

Top

 Post subject: Re: Extracting simple modelsPosted: Fri Jan 17, 2014 6:48 pm

Joined: Thu Jun 03, 2010 11:05 am
Posts: 42
Has thanked: 18 times
Have thanks: 3 times

Top

 Post subject: Re: Extracting simple modelsPosted: Sat Jan 18, 2014 2:23 am
 VIP member

Joined: Wed Nov 05, 2008 12:16 pm
Posts: 742
Has thanked: 1644 times
Have thanks: 363 times
I have the file and could upload it as last resort, but this really should be added to the Tools blog.

Top

 Post subject: Re: Extracting simple modelsPosted: Sat Jan 18, 2014 9:52 am
 M-M-M-Monster veteran

Joined: Fri Apr 20, 2012 9:24 am
Posts: 1995
Has thanked: 358 times
Have thanks: 991 times
thx - updated start post

--------------------------
(This is a text backup only for maybe future quotations.)

Workaround for formats which use shorts for the vertices and half floats for the uvs:

save the mesh in two ways, first as ShortAll then as HF_UV (or HF_all).
In the first obj file do a replacement of 'vt' by '#vt'.
In the (HF_UV) obj file do a replacement of 'v ' by '#v '. (<<< Don't forget the space here.)

Then copy all vts (without the face indices) into the (ShortAll) obj.

(Yep, this is ugly but should work.)

_________________
Extracting simple models: http://forum.xentax.com/viewtopic.php?f=29&t=10894
Make_H2O-ForzaHor3-jm9.zip

"We are Microsoft. You will be assimilated. Resistance is Futile."

Last edited by shakotay2 on Sun Apr 24, 2016 1:00 pm, edited 3 times in total.

Top

 Display posts from previous: All posts1 day7 days2 weeks1 month3 months6 months1 year Sort by AuthorPost timeSubject AscendingDescending
 Page 1 of 18 [ 259 posts ] Go to page 1, 2, 3, 4, 5 ... 18  Next

 All times are UTC + 1 hour

#### Who is online

Users browsing this forum: No registered users and 1 guest

 You cannot post new topics in this forumYou cannot reply to topics in this forumYou cannot edit your posts in this forumYou cannot delete your posts in this forumYou cannot post attachments in this forum

Search for:
 Jump to:  Select a forum ------------------ General Discussion    Holy Cow!    Game news    Game Tech Tips    Cool mods    Website Game Modding    Tutorials    3D/2D models    Audio file formats    Graphic file formats    Video file formats    Game Archive    Compressed files and methods    Code Talk    Game Localization    Savegames    Old posts    Retro games MultiEx Commander Tools & Programs    Game specific tools    General game tools