Page 1 of 3

Part way to finding a vertex header, need some pointers

Posted: Sun Jan 22, 2017 1:08 am
by St3ve
I'm trying to extract some of the wanzer models from Front Mission 4 but I'm having trouble locating their vertex header either through savestates or individual files. Originally, I couldn't grasp the details of model extraction and tried to leave the effort to the guy from PS23DFormat who did meet with some success- he identified the terrain vertex header and managed to create an unpacker for the games archive. After months of silence, though, it became clear that I couldn't simply sit back and would have to continue on my own. Thus far, I have gone through the xentax guides and cobbled together a basic python script that creates vertices from three consecutive values read in from a file but haven't managed to locate the vertex header for the mechs which I want to extract. I'm at a loss for what my next step should be- I tried to swap parts of the mechs around and then use a file comparison program but so many values changed it wasn't possible for me to locate a header, I tried poking around vertex clouds made form every variant of longs, shorts, and floats in hopes of working backwards but haven't seen anything recognizable. Should I go back to sifting through cheat engine for floats? Alter the range of my python script? Keep looking through the vertex clouds? Any ideas would be appreciated.

Re: Part way to finding a vertex header, need some pointers

Posted: Sun Jan 22, 2017 10:34 am
by shakotay2
welcome to the forum :)

1) could you show a picture of the terrain vertex header?
2) sample of a mech would be nice

Re: Part way to finding a vertex header, need some pointers

Posted: Mon Jan 23, 2017 1:39 am
by St3ve
The terrain vertex header is 0480*68 with * being the number of byte long floats that follow. The PS3D guy managed to find the face index as well but he didn't explain that part to me.
http://i.imgur.com/spyPAXG.jpg
http://i.imgur.com/rayVATj.jpg

Here's a snapshot of a mech from the savestate I've been working on, there are two more off screen. If you mean't an actual sample of code that's kind of a chicken and egg situation.
http://i.imgur.com/2TSKKp3.png

Here is the savestate itself if anyone wants to take a look.
https://drive.google.com/file/d/0B04lzj ... sp=sharing

Re: Part way to finding a vertex header, need some pointers

Posted: Mon Jan 23, 2017 7:20 pm
by shakotay2
reminds my of WildArms3 (http://ps23dformat.wikispaces.com/Wild+Arms+3)
I've tried the WildArms3eememoryto3ds.BMS with a modified search pattern (findloc OFFSET string "\x80\x00\x00\x00\xC0\x2E\x30\x12\x04\x00\x00\x00\x00\x00" 0 0 ; ) at no avail.

Produces 1 or two senseles 60 bytes files and I don't have the time to review that horrible script. :D

Re: Part way to finding a vertex header, need some pointers

Posted: Tue Jan 24, 2017 2:55 am
by St3ve
Right now, I' mulling over how to modify my vertex script- I wouldn't be surprised if the mech models used byte long floats like the terrain meshes but are obscured by all the non mesh vertices the script generates.

Re: Part way to finding a vertex header, need some pointers

Posted: Sat Jan 28, 2017 6:40 pm
by St3ve

Code: Select all

import bpy
import struct

coordinates = [0.0, 0.0, 0.0]
vertices = []

f = open(r"C:\Users\Greg\Documents\Front Mission savestates\eeMemory.bin", 'rb')

data = f.read(4)

while data != b'':
    value = struct.unpack('<f', data)
    value = value[0]

    if value <= 50 and value >= -50:
        coordinates[0] = value
        data = f.read(4)
        value = struct.unpack('<f', data)
        value = value[0]

        if value <= 50 and value >= -50:
            coordinates[1] = value
            data = f.read(4)
            value = struct.unpack('<f', data)
            value = value[0]

            if value <= 50 and value >= -50:
                coordinates[2] = value
                vertex = (coordinates[0], coordinates[1], coordinates[2])
                vertices.append(vertex)

    data = f.read(4)

mymesh = bpy.data.meshes.new("vertices")
myobject = bpy.data.objects.new("vertices", mymesh)
bpy.context.scene.objects.link(myobject)
mymesh.from_pydata(vertices,[],[])
Here's my script. It works but I just can't figure out how to filter code that isn't vertex data but still is a float, I tried having the program separate each "group" of vertices into an object but that slowed blender to a crawl. As It stands, I can't see past all these false positives but the only other option is to randomly test hundreds of individual files. Maybe opening a few of the files at a time rather than an entire savestate would provide better results.

Re: Part way to finding a vertex header, need some pointers

Posted: Sun Jan 29, 2017 6:48 am
by shakotay2
thx, but I don't have that bin file.

(used with eeMemory.bin the script fails:
"struct.error: unpack requires a bytes object of length 4")

Re: Part way to finding a vertex header, need some pointers

Posted: Sun Jan 29, 2017 3:48 pm
by St3ve
Sorry about that, I forgot to change the or operators to ands and repath it to the savestate I posted. It should "work" now.

Re: Part way to finding a vertex header, need some pointers

Posted: Sun Jan 29, 2017 6:04 pm
by shakotay2
thx; so your'e at the very beginning. :D
Why not split the eeMemory.bin into smaller parts and treating them seperately?

There's blocks of zeroes at 0x990BB0 to 0xE78360 (5 MB), another at 0x016E1280 to 0x01DC1300 (7 MB).

There's 70(?) TIM2 textures contained. Maybe identify/cut them first?

Re: Part way to finding a vertex header, need some pointers

Posted: Sun Jan 29, 2017 6:22 pm
by St3ve
I didn't know that was an option. Would there be a difference between this and testing the files I've extracted from the archive?

Re: Part way to finding a vertex header, need some pointers

Posted: Sun Jan 29, 2017 6:37 pm
by shakotay2
depends on what your extractor extracted. Did it create 70 TIM2 files?
If the extracted files in sum + 5 MB + 7 MB = 32 MB (=size of eeMemory.bin) I'd say: no.

(but that formulae is only applicable for uncompressed data)

Did you read here, for example? viewtopic.php?f=10&t=13680&p=113434&hilit=tim2#p113434

TextER.exe extracts 70 TIM2 files. (Pay attention that the parameter -tm to follow after the archivename (weird),
-e before.)
0028-TM2.JPG
This matches part of the point cloud your script is showing.

Re: Part way to finding a vertex header, need some pointers

Posted: Sun Jan 29, 2017 7:13 pm
by St3ve
The file extractor the PS23D guy made extracted about 3000 game files ranging from 0 to 300MB. While that sounds like a lot, the script generated quite a few duplicates I've been deleting on and off again. In any case, I poked around them in the beginning and found some level mesh data around the 1000kb mark- given how low poly the mesh data is, I would assume the mechs would be less than half that with weapons and backpacks being smaller still, which does narrow things down and cuts down on the false floats. I think I'll try taking a look at them again.

I actually tried munge explorer and timtool but they weren't as helpful as I thought. It's pretty hard to tell what textures go where without using texmod. If I ever manage to extract models out of Front Mission Online, it's not going to be fun using old screenshots from 2005 as a guide...

Re: Part way to finding a vertex header, need some pointers

Posted: Sun Jan 29, 2017 10:04 pm
by shakotay2
for the vertices shown by your script there's a more selective way to find them:
search for 0480xx68 (xx being a wildcard) in eeMemory.bin, there's 1523 finds.
After find address +4 log 11*3 floats each (11 vertices).
eeMemory-bin-Arena.JPG

Re: Part way to finding a vertex header, need some pointers

Posted: Sun Jan 29, 2017 10:26 pm
by St3ve
Performing a wildcard search for the terrain header was actually the first script I made but I don't understand what "+4 log 11*3 floats each (11 vertices)" means or does. Are you trying to exclude the terrain vertices?

Re: Part way to finding a vertex header, need some pointers

Posted: Mon Jan 30, 2017 12:17 am
by shakotay2
it just means that I logged 33 floats (11 vertices) starting 4 bytes after each pattern find address (1523 vertices)

ah, ok, re-read your posts, seems I misunderstood your problem.

Why exactly did you stick to a general search again after having performed a more specific one.
Wouldn't it make more sense to search the face indices instead?

Maybe they have to be autogenerated (guess tri strips).