XeNTaX Forum Index
Forum MultiEx Commander Tools Tools Home
It is currently Wed Sep 20, 2017 2:55 am

All times are UTC + 1 hour


Forum rules


Please click here to view the forum rules



Post new topic Reply to topic  [ 63 posts ]  Go to page 1, 2, 3, 4, 5  Next
Author Message
 Post subject: 3D model reversing compilation
PostPosted: Tue Aug 23, 2011 12:12 am 
Offline
M-M-M-Monster veteran
M-M-M-Monster veteran

Joined: Sat Apr 09, 2011 1:22 am
Posts: 2387
Has thanked: 170 times
Have thanks: 277 times

3D model formats index
3D model guide

Now something is coming together.

finale00 wrote:
We should probably add something similar to the definitive guide to exploring file formats, for 3D models.
The layout would be similar, in particular listing some common patterns that are usually observed. It would then diverge into different "types" of formats, as I would imagine models for console games would be much different from models for PC games for various reasons.

Though, I'm not sure if people actually check the wiki :|


You can make the ads go away by registering


_________________
Model Import Plugins/Scripts

Noesis Plugins | Reference files

Reference
Guide to 3D model reversal | 3D glossary


Last edited by finale00 on Thu Jan 12, 2012 7:42 pm, edited 2 times in total.

Top
 Profile  
 
 Post subject: Re: 3D model reversing compilation
PostPosted: Tue Jan 10, 2012 1:54 pm 
Offline
Banned

Joined: Sun Apr 10, 2011 5:30 pm
Posts: 95
Has thanked: 0 time
Have thanks: 14 times
Good idea!
Every 3d formats has some bare minimum things. vertices, primitives such as triangles and tristrips, materials, textures, matrices, bones, etc.

Any ideas how to start though?


Top
 Profile  
 
 Post subject: Re: 3D model reversing compilation
PostPosted: Tue Jan 10, 2012 6:23 pm 
Offline
M-M-M-Monster veteran
M-M-M-Monster veteran

Joined: Sat Apr 09, 2011 1:22 am
Posts: 2387
Has thanked: 170 times
Have thanks: 277 times
Rather than a tutorial approach, which requires you to know what you're actually talking about, maybe simply a listing of common things to look out for, what they represent, and what kind of clues you might see. Then someone else can add whatever they see fits.

The introduction of any 3D format would probably be a good start, like what a vertex is, what a face is, what materials are, etc.

Common vertex/index storage design.
Maybe some common file structures.

And then later on have some "techniques" on how to figure out the structure of a format.

_________________
Model Import Plugins/Scripts

Noesis Plugins | Reference files

Reference
Guide to 3D model reversal | 3D glossary


Top
 Profile  
 
 Post subject: Re: 3D model reversing compilation
PostPosted: Tue Jan 10, 2012 6:53 pm 
Offline
Banned

Joined: Sun Apr 10, 2011 5:30 pm
Posts: 95
Has thanked: 0 time
Have thanks: 14 times
I've been into 3d game programming so I could write about how things work internally and what values they can be represented with.

I don't have too much experience on common things to look for, you'll have to write most of that, until others join that is.

EDIT: Here is something for start:
Quote:
Please read the Definitive Guide to Exploring File Formats before starting this guide, which is aimed specifically for 3d file formats.

No knowledge in 3d is needed, however, any experience in 3d modelling or 3d programming will help. No advance math knowledge is used in 3d formats, though it is in 3d programming, but those are usually hidden inside 3d libraries you use, like your 3d engine or your 3d modeller's scripting language. You should be able to understand what vectors and matrixes are though.

We should first understand the basics of how 3d data is represented.

The building block of 3d data is the vertex. A vertex is a point in 3d space. So a vertex needs 3 values: x, y and z position. The position values are usually represented as 4 byte floats, though not always.

Two vertices (“vertices” is the plural for “vertex”) can connect and form a line.

Three vertices can connect and form a triangle. Although more vertices can connect and form more coplex polygons like quads (4 vertices), triangles are the most common. In fact, the GPU needs to break more complex polygons down to triangles before processing them, so 3d formats are likely to always use triangles.
But how are triangles stored in files? Three vertices (3*3 position values) are enough to create a triangle. But this is not how triangles are always generated from the vertex data in the 3d files.
Triangles can also be represented as what is called a “tristrip”, or triangle strip. In a triangle strip, the first three vertices form a triangle, then every new vertex creates a new triangle by connecting with the previous two.
Tristrips are performance optimization for some GPUs.

There are other ways to connect vertices, like linestrips or trifans (short for “triangle fan”, each new vertex after the 1st one creates a new triangle, by connecting with the previous and the very 1st vertex). However these are less common.

Some 3d formats will use what is called an “index buffer” (index list), together with the “vertex buffer” (vertex list). Index buffer is basically a list of (usually integer) numbers which tells which vertices connect with each other by their index in the vertex buffer.

A collection of triangles or tristrips is called a “Mesh” or “Surface”. Each file you explore will likely have more than one of these.

Now we have a basic understanding on how 3d graphics work.
GPU's main job is to process vertices. They are very good at it. It is possible to store and render 3d graphics using other methods, like voxels. But 99.9% of modern GPUs are only good for rendering 3d graphics from the data we described above and most games need the GPU to run, as CPU is too slow for rendering 3d graphics in realtime, polygonal or voxel based.


We still don't know enough about how 3d data is stored though. We talked about vertices, but we didn't talk about vertex colors, vertex normals, vertex UV maps, materials and other common data which is used in just about every game. We didn't talk much about vertex and index buffers and we also don't know how 3d animations work, what are bones and vertex weights and how they are represented.

...


Top
 Profile  
 
 Post subject: Re: 3D model reversing compilation
PostPosted: Tue Jan 10, 2012 7:49 pm 
Offline
M-M-M-Monster veteran
M-M-M-Monster veteran

Joined: Sat Apr 09, 2011 1:22 am
Posts: 2387
Has thanked: 170 times
Have thanks: 277 times
I've never worked with any 3D and the only experience I have is looking at the formats.

You'll probably have a better idea what the data would represent. That would be useful because if I were given a set of values stored in a particular location (eg: when defining the mesh), I wouldn't know what it could be.

So a glossary would be useful.

Anyways

http://wiki.xentax.com/index.php/3D_Model_Guide
http://wiki.xentax.com/index.php/3D_Model_Glossary

The guide would be the tutorial-like page, while the glossary is a list of stuff that engines might use.

The guide itself will be fairly simplistic. I mean, it's not that much different from trying to reverse any other format, and you can only go so far. An explanation of how all the stuff works would eventually take up the majority of the page.

The glossary will be the main focus, cause I can quickly identify a struct, but have no clue what the struct holds.

You can handle the structure of the glossary. Maybe you might group it by concept (mesh, material, animation, etc) or alphabetically (though that might not be as useful).

Some things that I would find useful:

Simple transformations like rotating, scaling, and transforming. How the vectors look and possibly why that is the case (though that is more mathematical so maybe it's not too important)

Bones and stuff. Never played around with 3D editor so never made a model. Not really sure how stuff ties in with one another.

Animation. There are keyframes I guess? That's all I know lol

_________________
Model Import Plugins/Scripts

Noesis Plugins | Reference files

Reference
Guide to 3D model reversal | 3D glossary


Top
 Profile  
 
 Post subject: Re: 3D model reversing compilation
PostPosted: Tue Jan 10, 2012 8:27 pm 
Offline
M-M-M-Monster veteran
M-M-M-Monster veteran

Joined: Sat Apr 09, 2011 1:22 am
Posts: 2387
Has thanked: 170 times
Have thanks: 277 times
lol 3Dmax costs $3500 for a license!!! :[

Noesis it is.

_________________
Model Import Plugins/Scripts

Noesis Plugins | Reference files

Reference
Guide to 3D model reversal | 3D glossary


Top
 Profile  
 
 Post subject: Re: 3D model reversing compilation
PostPosted: Tue Jan 10, 2012 8:35 pm 
Offline
Banned

Joined: Sun Apr 10, 2011 5:30 pm
Posts: 95
Has thanked: 0 time
Have thanks: 14 times
Blender is kind of cheap if you want.

Also "The role of the GPU" is not necessary. I just wanted to explain that modern GPUs work on vertices, so it is very very unlikely a game will use non-polygonal way of representing models (like voxels).


Top
 Profile  
 
 Post subject: Re: 3D model reversing compilation
PostPosted: Tue Jan 10, 2012 9:00 pm 
Offline
M-M-M-Monster veteran
M-M-M-Monster veteran

Joined: Sat Apr 09, 2011 1:22 am
Posts: 2387
Has thanked: 170 times
Have thanks: 277 times
I've tried writing an import script using blender with my Sanae 3D model converter and it sort of worked (I could import and render any model format I had written a plugin for successfully, but I couldn't apply textures), but then I got kind of lazy and just left it there (viewtopic.php?p=58523#p58523)

As someone that's probably only interested in reversing a model, I only need tools that do the bare minimum. So for example a model editor is a tool that comes with all sorts of features I don't need and doesn't make it any easier for me.

_________________
Model Import Plugins/Scripts

Noesis Plugins | Reference files

Reference
Guide to 3D model reversal | 3D glossary


Top
 Profile  
 
 Post subject: Re: 3D model reversing compilation
PostPosted: Wed Jan 11, 2012 9:52 am 
Offline
Banned

Joined: Sun Apr 10, 2011 5:30 pm
Posts: 95
Has thanked: 0 time
Have thanks: 14 times
I added everything I knew. There's nothing else I can add.

Why is this in the Holy Cow senction? Please repost in the 3D section.


Top
 Profile  
 
 Post subject: Re: 3D model reversing compilation
PostPosted: Wed Jan 11, 2012 10:20 am 
Offline
Site Admin
User avatar

Joined: Mon Sep 13, 2004 1:55 am
Posts: 703
Location: The wiki.
Has thanked: 107 times
Have thanks: 122 times
gjinka wrote:
Why is this in the Holy Cow senction? Please repost in the 3D section.


I moved the topic, and while I was at it, took the liberty of stickying it as well; if anyone disagrees with it being stickied for whatever reason, feel free to tell me. =)

_________________
Welcome to Xentax!

Rules | Requests | Wiki

If you run across a post that breaks the rules, please PM Craptain, Mr.Mouse, or myself - we will handle it from there.


Top
 Profile  
 
 Post subject: Re: 3D model reversing compilation
PostPosted: Wed Jan 11, 2012 10:52 pm 
Offline
M-M-M-Monster veteran
M-M-M-Monster veteran

Joined: Sat Apr 09, 2011 1:22 am
Posts: 2387
Has thanked: 170 times
Have thanks: 277 times
Now all we just need is a main 3D page and an index, and then it's all set up for creating pages for 3D formats and also linking them together.

http://wiki.xentax.com/index.php/3D_Model_Formats

Maybe someone can add this to the frontpage. It will be the index, with some other stuff at the top.

Also are there any standards for writing formats?
Should there be standards?

Also, in the event that people actually use the wiki to share information about various formats, there probably will need to be some sort of article naming standard.

For example, some game XYZ might have 5 different resource formats (model, sound, images, archive, save game). Some questions would immediately arise

1: should all of the information be grouped together under one article for that particular game? So the article would just be the name of the game.

2: Maybe we should have separate articles for each resource? Maybe something like XYZ_Audio. Mediawiki supports subpaging as well, so maybe we can just create an article called XYZ and then for any resources related to XYZ, we just toss them in a subpage.

_________________
Model Import Plugins/Scripts

Noesis Plugins | Reference files

Reference
Guide to 3D model reversal | 3D glossary


Top
 Profile  
 
 Post subject: Re: 3D model reversing compilation
PostPosted: Thu Jan 12, 2012 9:05 am 
Offline
Banned

Joined: Sun Apr 10, 2011 5:30 pm
Posts: 95
Has thanked: 0 time
Have thanks: 14 times
I don't know if creating a new page for only 3d models is a good idea.

And add the links to your first post.


Top
 Profile  
 
 Post subject: Re: 3D model reversing compilation
PostPosted: Thu Jan 12, 2012 7:40 pm 
Offline
M-M-M-Monster veteran
M-M-M-Monster veteran

Joined: Sat Apr 09, 2011 1:22 am
Posts: 2387
Has thanked: 170 times
Have thanks: 277 times
The GFFC's central theme is file formats, and I suppose listing all sorts of information regarding formats for various software.

I treat it like a DB, and would design the pages as such. If a user is wondering which game has 3D model formats available, and there's a page that lists all that information very straightforwardly, it is probably more useful than another page that lists all of the games that have *some* format available, and perhaps indicates which formats are available.

Grouping on resource is reasonable because there are typically only a few resources that make up some software, and typically only a few resources that people actually care about.

_________________
Model Import Plugins/Scripts

Noesis Plugins | Reference files

Reference
Guide to 3D model reversal | 3D glossary


Top
 Profile  
 
 Post subject: Re: 3D model reversing compilation
PostPosted: Thu Jan 12, 2012 9:20 pm 
Offline
Banned

Joined: Sun Apr 10, 2011 5:30 pm
Posts: 95
Has thanked: 0 time
Have thanks: 14 times
I think most people prefer format specifcations instead of scripts. You aren't expected to know 3 languages to understand the wiki. I understand why you made that page but I think people will think we are encouraging them to post a link to a script instead of writing proper specifications in the appropriate place.

BTW, guys please add some info to the wiki. All I wrote was from my experience of few 3d formats. It's nowhere complete.


Top
 Profile  
 
 Post subject: Re: 3D model reversing compilation
PostPosted: Thu Jan 12, 2012 9:44 pm 
Offline
M-M-M-Monster veteran
M-M-M-Monster veteran

Joined: Sat Apr 09, 2011 1:22 am
Posts: 2387
Has thanked: 170 times
Have thanks: 277 times
Each format page will contain specs, as well as links to existing scripts for those that just want scripts.
Similar to the archive formats.

I'm sure people aren't going to be as excited about writing documentation anyways 8D


_________________
Model Import Plugins/Scripts

Noesis Plugins | Reference files

Reference
Guide to 3D model reversal | 3D glossary


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 63 posts ]  Go to page 1, 2, 3, 4, 5  Next

All times are UTC + 1 hour


Who is online

Users browsing this forum: rusian and 9 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group