[Request]Hob game MDL converter

[Kva3imoda]

Hello,

I need your help with MDL 3models in the Hob game. Solving this problem would give us (Torchmodders.com and all community modders of Torchlight) new opportunities to create mods for the Torchlight/Torchlight2 games.

Firstly, if you want to unpack the Hob-archive you need this: Hob's Pak Extractor

Secondly, all "Runic Games" games are made on the OGRE engine, so you will need some tools: OgreXMLConverters + Ogre Meshy viewer

Problem description and tasks:
1. Previously, all "RG" games used 3models in Mesh format, but this Hob game uses a new MDL-format. As I found out, it's the same format, but slightly modified. These minor modifications do not allow us to use standard OGRE utilities.

a) I opened these Hob MDL files with Hex editor and found 2 types of files:



I do not know the difference, but it made me think that these are ordinary OGRE files. But when I tried to find this MeshSerializer_v1.9, I discovered that it does not exist. I found only 1.41, 1.8 and 1.10(see tools above). OgreSDK 1.9 contains 1.8 MeshSerializer.

b) I tried to rename these files MDL->MESH and tried to open in Ogre Meshy viewer but failed. OGRE does not understand this MeshSerializer.

c) Using the Hex editor, I changed this MeshSerializer_v1.9_o -> MeshSerializer_v1.100



After that you can open it in Ogre Meshy and convert this Mesh into a xml-file using OgreXMLConverters1.100(MeshSerializer_v1.100). Now using this xml-file you can convert the model to any MeshSerializer version.(Torchlight2 required 1.41).




Also, as I understood, the MDL file contains information about the material (maybe this is the cause of the main problem). But for convertible models it does not matter. OgreXMLConverter just ignores this and after the reconversion this information is lost.



Meanwhile. Hooray, it works I thought but...

d) Later I discovered that this simple method works only for a couple of models. For 99% of models this does not work. Examples of models in this archive: HOB_MODELS I divided them into convertible and non-convertible.
Also I added TLMeshViewer_v0.8.2.0 - utility to view models from the Torchlight games.

e) I'm looking for a way how to get the compatible MESH models. Any help is appreciated.

[akderebur]

The problem might be the difference in vertex block lengths.

The non-convertible models have 88 bytes vertex blocks. Here is batrabbit for example.



While the convertible ones have 32 bytes vertex blocks.



So this might be the reason why the model viewer fails. If you have the model viewer source, this should be an easy fix. Still I don't know the Ogre format at all, so it might very well be something else.

I would suggest you to try and find out the differences between convertible and non-convertible ones. Vertex block lenghts is the first thing I noticed, there might be other differences too.

[Kva3imoda]

The problem might be the difference in vertex block lengths.

The non-convertible models have 88 bytes vertex blocks. Here is batrabbit for example.

Oh, this is a great start. Thank you. Now we are one step closer to the final goal. Now I see that there are more differences than I thought.

[shakotay2]

here is a model from nonconvertible_v1.9_o with FVF size of 32:
what exactly does happen (text of error message for example?) when you change the MeshSerializer_v1.9_o -> MeshSerializer_v1.100 for C_Frogbeast.mdl and try to load it into Qgre Meshy?

The mesh format itself looks simple:

C_Frogbeast.jpg

well, see, akderebur was quicker...

[Kva3imoda]

when you change the MeshSerializer_v1.9_o -> MeshSerializer_v1.100 for C_Frogbeast.mdl and try to load it into Qgre Meshy?

Well, nothing happens.

LOG:
Creating resource group InternalPermanentMeshGroup
Added resource location 'Resources/Models' of type 'FileSystem' to resource group 'InternalPermanentMeshGroup'
Added resource location 'Resources/Fonts' of type 'FileSystem' to resource group 'InternalPermanentMeshGroup'
Added resource location 'Resources/RTShaderLib' of type 'FileSystem' to resource group 'InternalPermanentMeshGroup'
Added resource location 'Resources/RTShaderLib/GLSL' of type 'FileSystem' to resource group 'InternalPermanentMeshGroup'
Added resource location 'Resources/RTShaderLib/GLSLES' of type 'FileSystem' to resource group 'InternalPermanentMeshGroup'
Added resource location 'Resources/RTShaderLib/HLSL' of type 'FileSystem' to resource group 'InternalPermanentMeshGroup'
Added resource location 'Resources/RTShaderLib/Cg' of type 'FileSystem' to resource group 'InternalPermanentMeshGroup'
Initialising resource group InternalPermanentMeshGroup
Parsing scripts for resource group InternalPermanentMeshGroup
Parsing script Axis.material
Parsing script BoneMesh.material
Parsing script Grid.material
Finished parsing scripts for resource group InternalPermanentMeshGroup
Creating resources for group InternalPermanentMeshGroup
All done
Creating resource group ShaderGeneratorResourceGroup
Added resource location 'C:/Users/****/AppData/Roaming/OgreMeshy/RTShaderCache/' of type 'FileSystem' to resource group 'ShaderGeneratorResourceGroup'
Creating resource group InternalMeshGroup
Destroying resource group InternalMeshGroup
Unloading resource group InternalMeshGroup
Finished unloading resource group InternalMeshGroup
Creating resource group InternalMeshGroup
Added resource location 'F:\HOB\Xentax\MODELS\nonconvertible_v1.9_o\C_FROGBEAST' of type 'FileSystem' to resource group 'InternalMeshGroup'
Parsing scripts for resource group Autodetect
Finished parsing scripts for resource group Autodetect
Creating resources for group Autodetect
All done
Parsing scripts for resource group General
Finished parsing scripts for resource group General
Creating resources for group General
All done
Parsing scripts for resource group Internal
Finished parsing scripts for resource group Internal
Creating resources for group Internal
All done
Parsing scripts for resource group InternalMeshGroup
Finished parsing scripts for resource group InternalMeshGroup
Creating resources for group InternalMeshGroup
All done
Parsing scripts for resource group ShaderGeneratorResourceGroup
Finished parsing scripts for resource group ShaderGeneratorResourceGroup
Creating resources for group ShaderGeneratorResourceGroup
All done
Mesh: Loading C_FROGBEAST.MESH.
Texture: Grid.png: Loading 1 faces(PF_A8R8G8B8,32x32x1) Internal format is PF_A8R8G8B8,32x32x1.

And LOG for the convertible model C_BIRD:

Creating resource group InternalPermanentMeshGroup
Added resource location 'Resources/Models' of type 'FileSystem' to resource group 'InternalPermanentMeshGroup'
Added resource location 'Resources/Fonts' of type 'FileSystem' to resource group 'InternalPermanentMeshGroup'
Added resource location 'Resources/RTShaderLib' of type 'FileSystem' to resource group 'InternalPermanentMeshGroup'
Added resource location 'Resources/RTShaderLib/GLSL' of type 'FileSystem' to resource group 'InternalPermanentMeshGroup'
Added resource location 'Resources/RTShaderLib/GLSLES' of type 'FileSystem' to resource group 'InternalPermanentMeshGroup'
Added resource location 'Resources/RTShaderLib/HLSL' of type 'FileSystem' to resource group 'InternalPermanentMeshGroup'
Added resource location 'Resources/RTShaderLib/Cg' of type 'FileSystem' to resource group 'InternalPermanentMeshGroup'
Initialising resource group InternalPermanentMeshGroup
Parsing scripts for resource group InternalPermanentMeshGroup
Parsing script Axis.material
Parsing script BoneMesh.material
Parsing script Grid.material
Finished parsing scripts for resource group InternalPermanentMeshGroup
Creating resources for group InternalPermanentMeshGroup
All done
Creating resource group ShaderGeneratorResourceGroup
Added resource location 'C:/Users/****/AppData/Roaming/OgreMeshy/RTShaderCache/' of type 'FileSystem' to resource group 'ShaderGeneratorResourceGroup'
Creating resource group InternalMeshGroup
Destroying resource group InternalMeshGroup
Unloading resource group InternalMeshGroup
Finished unloading resource group InternalMeshGroup
Creating resource group InternalMeshGroup
Added resource location 'F:\HOB\Xentax\MODELS\convertible\C_BIRD' of type 'FileSystem' to resource group 'InternalMeshGroup'
Parsing scripts for resource group Autodetect
Finished parsing scripts for resource group Autodetect
Creating resources for group Autodetect
All done
Parsing scripts for resource group General
Finished parsing scripts for resource group General
Creating resources for group General
All done
Parsing scripts for resource group Internal
Finished parsing scripts for resource group Internal
Creating resources for group Internal
All done
Parsing scripts for resource group InternalMeshGroup
Finished parsing scripts for resource group InternalMeshGroup
Creating resources for group InternalMeshGroup
All done
Parsing scripts for resource group ShaderGeneratorResourceGroup
Finished parsing scripts for resource group ShaderGeneratorResourceGroup
Creating resources for group ShaderGeneratorResourceGroup
All done
Mesh: Loading C_BIRD_v1.100_Original.MESH.
Skeleton: Loading MEDIA/CHARACTERS/C_BIRD/C_BIRD.skeleton
OGRE EXCEPTION(6:FileNotFoundException): Cannot locate resource MEDIA/CHARACTERS/C_BIRD/C_BIRD.skeleton in resource group InternalMeshGroup or any other group. in ResourceGroupManager::openResource at G:/SDK/OgreLatest/OgreMain/src/OgreResourceGroupManager.cpp (line 757)
Unable to load skeleton MEDIA/CHARACTERS/C_BIRD/C_BIRD.skeleton for Mesh C_BIRD_v1.100_Original.MESH. This Mesh will not be animated. You can ignore this message if you are using an offline tool.
Can't assign material c_bird_mat_DELETEME to SubEntity of MeshEntity because this Material does not exist. Have you forgotten to define it in a .material script?
Texture: Grid.png: Loading 1 faces(PF_A8R8G8B8,32x32x1) Internal format is PF_A8R8G8B8,32x32x1.

[Kva3imoda]

Maybe for MeshSerializer_v1.9_o models we can find out why they are not converted?
Can we parse the structure of binary files and find the differences?
All files have a clear structure.

[Kva3imoda]

Ok, it looks like I've found a way to beat these "nonconvertible_v1.9_o" models.
1. I copied the beginning from another model C_BIRD_v1.100_Reconverted.MESH
from start to vertex count
2. Deleted the beginning in "nonconvertible_v1.9_o" C_FROGBEAST.MESH
from start to vertex count
3. Paste the beginning of one file to another.
Ta-da! Now this is being converted.

[Kva3imoda]

The second "nonconvertible_v1.9_o" model from example.



It looks like this method works.

now we need to beat the "nonconvertible_v1.9" files! Any ideas?

[Kva3imoda]

Another model for the test. This "copy-paste" method works fine for "nonconvertible_v1.9_o" models.

[Kva3imoda]

The problem might be the difference in vertex block lengths.

The non-convertible models have 88 bytes vertex blocks. Here is batrabbit for example.

Ok, I have a question. Can we convert these 88 bytes blocks into 32 bytes blocks? In theory?

[akderebur]

I will take a wild guess here. I think only "1.9_o" works because it has the same length as "1.100", 5 bytes each. So when you change "1.9" to "1.100" you are swifting all the positions by 2 bytes.

Like I said I don't know how this format works, but it seemed like a possibility to me. Try deleting 2 bytes from some part that isn't necessary to see if it will make a difference.

[Kva3imoda]

Try deleting 2 bytes from some part that isn't necessary to see if it will make a difference.

I've tried this before. This does not work. This "material" part, we can cut it as we wish.

[akderebur]

Ok, I have a question. Can we convert these 88 bytes blocks into 32 bytes blocks? In theory?

It is possible to write a script/program that takes the first 32 bytes of each block and deletes the rest, if that is what you mean by converting.

[Kva3imoda]

It is possible to write a script/program that takes the first 32 bytes of each block and deletes the rest

These 56 bytes are empty? The necessary information will be preserved?
It might work.

[akderebur]

These 56 bytes are empty? The necessary information will be preserved?
It might work.

They are not empty, so you might lose some data, but it is also possible that the data in those bytes are not necessary for ogre tools or torchlight. I can't say anything for sure since I don't know what those bytes represent.

Anyway here is the batrabbit model with 32 bytes blocks. You can give it a try at least.

batrab.rar