XeNTaX Forum Index
Forum MultiEx Commander Tools Tools Home
It is currently Mon Jun 18, 2018 8:15 pm

All times are UTC + 1 hour


Forum rules


Please click here to view the forum rules



Post new topic Reply to topic  [ 12 posts ] 
Author Message
 Post subject: Noesis / Maxscript scripting questions
PostPosted: Thu Jan 04, 2018 3:54 pm 
Offline
beginner

Joined: Tue Oct 13, 2015 10:25 am
Posts: 24
Has thanked: 3 times
Have thanks: 0 time














You can make the ads go away by registering

Hi all, i recently started getting my hands on Noesis and i wanted to give it a try to open some meshes and writing my own scripts to perform the task...
But i'm really new to it and i'm stuck at reading the binary file's vertices and faces i have an error message saying that the "vertex position list is empty" i tried looking at already existing scripts for other games assets but it's a bit confusing.
I mainly use Hex2obj and ModelResearcher to extract some meshes but having a script for doing it is faster and could help me for learning purposes...
Here is my script for now (i've made it with the exemple sample provided in Neosis) :
- http://www.mediafire.com/file/zh3diy5aj ... -Script.py

And this is the error message :
Image

Theses are the samples i'm using for making the script :
- http://www.mediafire.com/file/bddsn7szqrt8whs/%u7CD6%u679C%u821E%u53F0.rgm (STAGE file)
- http://www.mediafire.com/file/hiuk6edvh ... ancer1.rgm (AVATAR file)
- http://www.mediafire.com/file/nym1t18lt ... ove_01.rgm (ANIMATION file)

Note that my goal is mainly being able to read / import Avatars and Stages RGMs on Noesis (Animations aren't for me at all it's too difficult)
I've studied the format for a while now and i have collected many infos about it, i still have trouble saying what data types i'm seeing but i guess it will comes with time...

But after looking at many RGM samples i can give these informations (with help of other skilled peoples) :

Code:
File structure :
- little endian
- byte = 8 bit int
- short = 16 bit int
- long = 32 bit int
- float = 32 bit float


It's common to every RGMs i've "worked" with !
also it's a "dynamic file format" i don't know yet what it exactly means but i don't think it's really important for now...
Anyway thanks for reading, if someone can guide me with Noesis scripting i will really appreciate it.
I know that their is a tutorial made by "Chroxx" somewhere on xentax but it's missing the pictures (because links are dead) and it's still a bit confusing.
As usual sorry for my english.

Oh and Happy new year :]


Last edited by Loginoux on Wed Jan 10, 2018 1:45 pm, edited 1 time in total.

Top
 Profile  
 
 Post subject: Re: Noesis python script problem
PostPosted: Thu Jan 04, 2018 11:56 pm 
Offline
VIP member
VIP member
User avatar

Joined: Wed Nov 05, 2008 12:16 pm
Posts: 1091
Has thanked: 2138 times
Have thanks: 603 times
Loginoux wrote:
i tried looking at already existing scripts for other games assets but it's a bit confusing.

indeed many are confusing, and some for no reason, i newbified your script to open the discoclub_dancer1.rgm :D
Code:
from inc_noesis import *

def registerNoesisTypes():
    handle = noesis.register("Audition online dance battle [PC]", ".RGM")
    noesis.setHandlerTypeCheck(handle, noepyCheckType)
    noesis.setHandlerLoadModel(handle, noepyLoadModel)
    #noesis.logPopup()
    return 1

#check if it's this type based on the data
def noepyCheckType(data):
    bs = NoeBitStream(data)
    if noeStrFromBytes(bs.readBytes(9)) != "Delight3D": return 0
    return 1

#load the model
def noepyLoadModel(data, mdlList):              #standard line
    ctx = rapi.rpgCreateContext()               #standard line
    bs = NoeBitStream(data)                     #standard line
    bs.seek(0x25a, NOESEEK_ABS)      #jump to position 0x25a from 0x0
    vCount = bs.readUInt()            #read vertex count
    bs.readByte()                      #skip this unknown byte
    vBuff = bs.readBytes(vCount * 32)   #read vertex buffer which is vertex count * vertex stride
    fCount = bs.readUInt() * 3           #read face count and multiply by 3
    fBuff = bs.readBytes(fCount * 2)      #read face buffer which is face count * 2
    rapi.rpgBindPositionBufferOfs(vBuff, noesis.RPGEODATA_FLOAT, 32, 0)   #pos of vertices (byte array, data type, stride, pos within stride)
    rapi.rpgBindUV1BufferOfs(vBuff, noesis.RPGEODATA_FLOAT, 32, 24)       #UV1  (byte array, data type, stride, pos within stride)
    rapi.rpgCommitTriangles(fBuff, noesis.RPGEODATA_SHORT, fCount, noesis.RPGEO_TRIANGLE, 1)
    mdlList.append(rapi.rpgConstructModel())    #standard line
    rapi.rpgClearBufferBinds()                  #standard line
    return 1                                    #standard line

see how simple it can be? this script reads only the mesh and UVs but still, its simple. :D
i put comments in there so you can see what is happening.

edit
adding link to original research thread because it has much info about basic rgm reversal already (:
viewtopic.php?f=16&t=14885

_________________
Noesis
Hex2obj
QuickBMS
Unity Studio
TextureFinder


Last edited by AceWell on Tue Mar 13, 2018 1:48 am, edited 2 times in total.

Top
 Profile  
 
 Post subject: Re: Noesis python script problem
PostPosted: Fri Jan 05, 2018 1:37 am 
Offline
beginner

Joined: Tue Oct 13, 2015 10:25 am
Posts: 24
Has thanked: 3 times
Have thanks: 0 time
AceWell wrote:
Loginoux wrote:
i tried looking at already existing scripts for other games assets but it's a bit confusing.

indeed many are confusing, and some for no reason, i newbified your script to open the discoclub_dancer1.rgm :D
Code:
from inc_noesis import *

def registerNoesisTypes():
    handle = noesis.register("Audition online dance battle [PC]", ".RGM")
    noesis.setHandlerTypeCheck(handle, noepyCheckType)
    noesis.setHandlerLoadModel(handle, noepyLoadModel)
    #noesis.logPopup()
    return 1

#check if it's this type based on the data
def noepyCheckType(data):
    bs = NoeBitStream(data)
    magic = noeStrFromBytes(bs.readBytes(9))
    if magic != "Delight3D":
       return 0
    return 1

#load the model
def noepyLoadModel(data, mdlList):              #standard line
    ctx = rapi.rpgCreateContext()               #standard line
    bs = NoeBitStream(data)                     #standard line
    bs.seek(0x25a, NOESEEK_ABS)      #jump to position 0x25a from 0x0
    vCount = bs.readUInt()            #read vertex count
    bs.readByte()                      #skip this unknown byte
    vBuff = bs.readBytes(vCount * 32)   #read vertex buffer which is vertex count * vertex stride
    fCount = bs.readUInt() * 3           #read face count and multiply by 3
    fBuff = bs.readBytes(fCount * 2)      #read face buffer which is face count * 2
    rapi.rpgBindPositionBufferOfs(vBuff, noesis.RPGEODATA_FLOAT, 32, 0)   #pos of vertices (byte array, data type, stride, pos within stride)
    rapi.rpgBindUV1BufferOfs(vBuff, noesis.RPGEODATA_FLOAT, 32, 24)       #UV1  (byte array, data type, stride, pos within stride)
    if fCount % 3 == 0:                       #check if evenly divisible by 3 and do this
        rapi.rpgCommitTriangles(fBuff, noesis.RPGEODATA_SHORT, fCount, noesis.RPGEO_TRIANGLE, 1)
    else:                                        #if not evenly divisible by 3 do this instead
        rapi.rpgCommitTriangles(fBuff, noesis.RPGEODATA_SHORT, fCount, noesis.RPGEO_TRIANGLE_STRIP, 1)
    mdlList.append(rapi.rpgConstructModel())    #standard line
    rapi.rpgClearBufferBinds()                  #standard line
    return 1                                    #standard line

see how simple it can be? this script reads only the mesh and UVs but still, its simple. :D
i put comments in there so you can see what is happening.


Hi thank you for replying !
It's super clear now it makes more sense to me, i will try to read other RGM files and use this script as a base, i might need to change just some lines i guess !
I will update my progress here so far.
Thank you very much !


Top
 Profile  
 
 Post subject: Re: Noesis python script problem
PostPosted: Tue Jan 09, 2018 4:51 pm 
Offline
beginner

Joined: Tue Oct 13, 2015 10:25 am
Posts: 24
Has thanked: 3 times
Have thanks: 0 time
Hello guys, i tried to continue with Noesis scripting and i'm fine with it i can read some models now !
recently i tried to import the stage's RGM to 3ds max, so i started writing a script for it :
Code:
-- Clear listener
clearListener()
-- Open a "*.RGM" file
fRGM = getOpenFileName \
caption:"Open an RGM file:" \
filename:"C:\Users\RGM\Acv tool -CN\*.rgm" \
types:"RGM(*.rgm)|*.rgm"
-- Read binaries from "*.RGM"
if fRGM != undefined then
(
   f = fopen fRGM "rb"
   clearlistener()
   
   Vert_array=#()
   Face_array=#()
   UV_array=#()

   fseek f 762 #seek_set
   
   -- Vertex data
   
      vertexCount = readbyte f #unsigned
    print ("VertexCount: " + vertexCount as string)
   for x = 1 to vertexCount do
   (
      vx = readfloat f
      print ("VectorX: " + vx as string)
      vy = readfloat f
      vz = readfloat f
      append Vert_array[vx,vy,vz]
   )

      Print ("Last Read @ 0x"+((bit.intAsHex(ftell f))as string))
         
-- Close the RGM file
fclose f
      )


But i have a problem with vertex count... i think i have found the spot where it is stored in the file (i mean the address) but i have trouble actually reading the data...
i tried doing readlong, readfloats, readshort to get the right value (because i was confused and still having trouble telling if the data is a long short etc) but it's still not correct ! my script is stopping waaaay before the end of the vertex buffer...
Anyone have an idea ?

thanks.

PS : the vertex count address is right before the vertex buffer one in every file i saw (including avatars)
And also the file is a chinese one it's the same structure between all of the them just different another language...


Top
 Profile  
 
 Post subject: Re: Noesis / Maxscript scripting questions
PostPosted: Sun Feb 11, 2018 11:16 pm 
Offline
beginner

Joined: Tue Oct 13, 2015 10:25 am
Posts: 24
Has thanked: 3 times
Have thanks: 0 time
Hi guys !
After a few weeks i updated my script but i didn't managed to extract the stage mesh yet...
I have mainly collected infos about the format and updated my kind of "data structure" a little bit...
I discovered that 1 single vertex is stored as 32 bytes and that the 12 first bytes are basically x,y and z value the 20 others are still unknown to this date :/
So i tried skipping these "20" bytes left in my maxscript but it doesn't work i don't know how i could do it "manually"

Code:
-- Clear listener
clearListener()
-- Open a "*.RGM" file
fRGM = getOpenFileName \
caption:"Open an RGM file:" \
filename:"C:\Users\Videos\Acv tool -CN\*.rgm" \
types:"RGM(*.rgm)|*.rgm"

-- Read binaries from "*.RGM"
if fRGM != undefined then
(
   f = fopen fRGM "rb"
   clearlistener()
   
   Vert_array=#()
   Face_array=#()
   UV_array=#()

   --Header
   /*fseek f 0 #seek_set
   ID = readstring f
   print("ID: " + ID)
   fseek f 30 #seek_set
   unk01 = readbyte f
   print("unk01 = " + unk01 as string)
   fseek f 31 #seek_set
   unk02 = readbyte f
   print("unk02 = " + unk02 as string)
   fseek f 38 #seek_set
   unk03 = readbyte f
   print("unk03 = " + unk03 as string)
   fseek f 41 #seek_set
   unk04 = readstring f
   print("unk04 = " + unk04)
   fseek f 47 #seek_set
   unk05 = readshort f
   print("unk05 = " + unk05 as string)*/
   
   /*for x = 1 to 20 do(
      strings =readstring f
      whitespace = readshort f
      print("String:" + strings)
      print("Whitespace:" + whitespace as string)
   )*/
   
   -- Vertex data
   fseek f 0x2FA #seek_set
   vertexCount = readshort f #seet_set
   print ("VertexCount: " + vertexCount as string)
   for x = 1 to vertexCount do
   (
      vx = readfloat f
      vy = readfloat f
      vz = readfloat f
      -- Unknowns values
      --unk000 = readfloat f
      --unk001 = readfloat f
      --unk002 = readfloat f
     --unk003 = readfloat f
     --unk004 = readfloat f
    
      append Vert_array[vx,vy,vz]
   )
   
   -- face data
   /*fseek f 0x1EB83  #seek_set
   FacesCount = readshort f #unsigned
   print ("FacesCount: " + FacesCount as string)

   for i = 1 to FacesCount do
   (
      fa = readshort f #unsigned
      fb = readshort f #unsigned
      fc = readshort f #unsigned
      append Face_array[fa+1,fb+1,fc+1]
   )*/
   
   -- UV data (not any info in the file....)
   /*count = readlong f #unsigned
   for i = 1 to count do
   {
      tu = readfloat f
      tv = readfloat f
      append UV_array[tu,1-tv,0]
   )*/
   
   -- Build the mesh
   msh = mesh vertices:Vert_array faces:Face_array name:(getFilenameFile fRGM)
   --msh.numTVerts = UV_array.count
   --buildTVFaces msh
   --for j = 1 to UV_array.count do setTVert msh j UV_array[j]
   --for j = 1 to Face_array.count do setTVFace msh j Face_array[j]
   
   --Printing
   Print ("Last Read @ 0x"+((bit.intAsHex(ftell f))as string))
      
   free Face_array
   free Vert_array
   --free UV_array      
      
-- Close the RGM file
fclose f
      )


It's possible that i'm doing huge mistakes because i'm still beginner in this...
Thanks for reading if someone have any suggestion or tips feel free to let me know i will appreciate the help.


Last edited by Loginoux on Sun Mar 18, 2018 6:34 pm, edited 1 time in total.

Top
 Profile  
 
 Post subject: Re: Noesis / Maxscript / hex2obj questions
PostPosted: Mon Mar 12, 2018 11:23 am 
Offline
beginner

Joined: Tue Oct 13, 2015 10:25 am
Posts: 24
Has thanked: 3 times
Have thanks: 0 time
Hi, since i wasn't able to fix my last maxscript i tried alternative ways to export some meshes and i have managed to have some quite "good" results for exemple :

Image
Image
Image

I can export 100% of the vertices from any RGM now without any issues the only thing that bothers me are the faces... sometimes they are screwed and in some cases everything is fine i don't understand why, i'm using both ModelResearcher and Hex2obj for that...
If someone is interested i can provide some "problematic" samples cause it's really weird.

Also I don't have any new things on the structure of the format i just know the organization some bytes are still unknown to me more specifically for avatars... i have to remove these bytes and check how the game react to it maybe it will give me some ideas.

Anyway if someone have an idea for my maxscript feel free to let me know !
Have a nice day.


Top
 Profile  
 
 Post subject: Re: Noesis / Maxscript / hex2obj questions
PostPosted: Mon Mar 12, 2018 3:03 pm 
Offline
mega-veteran
mega-veteran
User avatar

Joined: Tue Jul 05, 2016 9:37 am
Posts: 226
Has thanked: 15 times
Have thanks: 87 times
Loginoux wrote:
I can export 100% of the vertices from any RGM now without any issues the only thing that bothers me are the faces... sometimes they are screwed and in some cases everything is fine i don't understand why, i'm using both ModelResearcher and Hex2obj for that...

Normally it's because the face indices chunk is a group of submeshes referencing vertices starting from somewhere else at the vertices chunk.
There should be some fields in the file telling you all the submeshes' indicies counts or something like that. Same thing can be applied to the vertices.

_________________
An Imitable Workflow for Reverse Engineering a Game Model

"No investigation means no right to speak."
Remember to thank people when you get helped. That's the minimum respect to their efforts.


Top
 Profile  
 
 Post subject: Re: Noesis / Maxscript / hex2obj questions
PostPosted: Mon Mar 12, 2018 4:22 pm 
Offline
beginner

Joined: Tue Oct 13, 2015 10:25 am
Posts: 24
Has thanked: 3 times
Have thanks: 0 time
Bigchillghost wrote:
Loginoux wrote:
I can export 100% of the vertices from any RGM now without any issues the only thing that bothers me are the faces... sometimes they are screwed and in some cases everything is fine i don't understand why, i'm using both ModelResearcher and Hex2obj for that...

Normally it's because the face indices chunk is a group of submeshes referencing vertices starting from somewhere else at the vertices chunk.
There should be some fields in the file telling you all the submeshes' indicies counts or something like that. Same thing can be applied to the vertices.


Hum interesting, after my investigations (it could be 200% wrong) of the format i've noticed it's mainly something like : vertex count -> vertex buffer / Face count -> face buffer and when their is multiples meshes into 1 file it's just : [Object 1] vertex buffer / face buffer / texture string [Object 2] vertex buffer / face buffer... etc.

So i don't know if it could be the issue here, but it might be.
I just have to know how to read the faces correctly with hex2obj now that you informed me about that.


Top
 Profile  
 
 Post subject: Re: Noesis / Maxscript / hex2obj questions
PostPosted: Tue Mar 13, 2018 1:01 am 
Offline
mega-veteran
mega-veteran
User avatar

Joined: Tue Jul 05, 2016 9:37 am
Posts: 226
Has thanked: 15 times
Have thanks: 87 times
Loginoux wrote:
I just have to know how to read the faces correctly with hex2obj now that you informed me about that.

You have to find all the needed elements(counts & offsets) for each submesh, that's how hex2obj works.

_________________
An Imitable Workflow for Reverse Engineering a Game Model

"No investigation means no right to speak."
Remember to thank people when you get helped. That's the minimum respect to their efforts.


Top
 Profile  
 
 Post subject: Re: Noesis / Maxscript scripting questions
PostPosted: Fri Mar 16, 2018 4:17 pm 
Offline
beginner

Joined: Tue Oct 13, 2015 10:25 am
Posts: 24
Has thanked: 3 times
Have thanks: 0 time
Bigchillghost wrote:
Loginoux wrote:
I just have to know how to read the faces correctly with hex2obj now that you informed me about that.

You have to find all the needed elements(counts & offsets) for each submesh, that's how hex2obj works.


Let's take an exemple...
Here is the problematic file : http://www.mediafire.com/file/h33mj31tf ... Fa3703.rgm

At first i'm trying to have a "point cloud" from the file i'm working with, in the case of this rgm everything is alright as you can see on the picture the values i entered seems correct.

Image

But when it comes to faces (note that this is a single "mesh" RGM, every vertices are stored into 1 single vertex buffer) they are screwed !

Image

Maybe i didn't understood what you were trying to explain to me ? i don't know...
Also i have trouble finding the UVs, some parts of the file are still weird for me especially what's after the face's short values array (at the end of the textures path strings).


Top
 Profile  
 
 Post subject: Re: Noesis / Maxscript scripting questions
PostPosted: Fri Mar 16, 2018 7:54 pm 
Offline
M-M-M-Monster veteran
M-M-M-Monster veteran
User avatar

Joined: Fri Apr 20, 2012 9:24 am
Posts: 2265
Location: Nexus, searching for Jim Kirk
Has thanked: 510 times
Have thanks: 1172 times
sometimes you've to use less face indices, then look where to proceed with the next "submesh" even if you think there aren't any:


Attachments:


You do not have the required permissions to view the files attached to this post. Register to gain access.


_________________
"you can't always get things served on a silver tray"
extracting simple models: http://forum.xentax.com/viewtopic.php?f=29&t=10894
Make_H2O-ForzaHor3-jm9.zip

"We are Microsoft. You will be assimilated. Resistance is Futile."


Top
 Profile  
 
 Post subject: Re: Noesis / Maxscript scripting questions
PostPosted: Sat Mar 17, 2018 1:57 am 
Offline
mega-veteran
mega-veteran
User avatar

Joined: Tue Jul 05, 2016 9:37 am
Posts: 226
Has thanked: 15 times
Have thanks: 87 times
Loginoux wrote:
Maybe i didn't understood what you were trying to explain to me ? i don't know...

This is what I mean:
To make it brief, I'll just skip those normals and UV stuffs. Say there's a file containing two submeshes where geo data are arranged like:

Code:
submesh 1      some other data . . .         submesh 2
               then comes submesh 2      
v x11 y11 z11                                 v x21 y21 z21
v x12 y12 z12                                 v x22 y22 z22
v x13 y13 z13                                 v x23 y23 z23
...                                          ...

f v11 v12 v13                                 f v21 v22 v23
...                                          ...

For such case you can load these two submeshes one by one easily. But what if I put them together like this without updating the values of the indices of submesh 2?
Code:
v x11 y11 z11
v x12 y12 z12
v x13 y13 z13
...
v x21 y21 z21
v x22 y22 z22
v x23 y23 z23
...

f v11 v12 v13
...
f v21 v22 v23
...

In this case you can't load it as one single mesh.
But obviously it has nothing to do with your condition. Here you're just making a mistake.
The triangle count before the indices buffer is a LONG32 interger.


Attachments:


You do not have the required permissions to view the files attached to this post. Register to gain access.


_________________
An Imitable Workflow for Reverse Engineering a Game Model

"No investigation means no right to speak."
Remember to thank people when you get helped. That's the minimum respect to their efforts.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 12 posts ] 

All times are UTC + 1 hour


Who is online

Users browsing this forum: daemon1, lainfiquette and 6 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group