Noesis tutorial Basic Model

Read or post any tutorial related to file format analysis for modding purposes.
User avatar
MaZTeR
veteran
Posts: 140
Joined: Sat Jul 09, 2016 4:06 pm
Location: Finland
Has thanked: 1 time
Been thanked: 2 times

Re: Noesis tutorial Basic Model

Post by MaZTeR » Tue Oct 18, 2016 5:55 pm

The download link doesn't work. I really need this tool ASAP :)

User avatar
eri619
advanced
Posts: 79
Joined: Wed May 16, 2012 6:36 am
Location: India
Has thanked: 11 times
Been thanked: 1 time
Contact:

Re: Noesis tutorial Basic Model

Post by eri619 » Sun Nov 06, 2016 6:20 am

please re upload the images,tutorial without images are pretty useless.No offence.

dragoncrest
ultra-n00b
Posts: 5
Joined: Sun Sep 21, 2014 4:47 pm
Has thanked: 6 times

Re: Noesis tutorial Basic Model

Post by dragoncrest » Sun May 13, 2018 4:07 pm

can you make a tutorial about animation?
thanks.

User avatar
Tosyk
double-veteran
double-veteran
Posts: 936
Joined: Thu Oct 22, 2009 10:24 am
Location: Russia, Siberia
Has thanked: 226 times
Been thanked: 107 times
Contact:

Re: Noesis tutorial Basic Model

Post by Tosyk » Fri Jul 20, 2018 10:02 pm

in noesis, how to skip junk data like texture path at the beginning of If the length of this path is before it?
Thank you for all you do here
my blog | my forum

User avatar
shakotay2
MEGAVETERAN
MEGAVETERAN
Posts: 2612
Joined: Fri Apr 20, 2012 9:24 am
Location: Nexus, searching for Jim Kirk
Has thanked: 642 times
Been thanked: 1339 times

Re: Noesis tutorial Basic Model

Post by shakotay2 » Fri Jul 20, 2018 11:05 pm

bs = NoeBitStream(data)
lenStr = bs.readUShort()# 2 bytes for length of pathstring
bs.seek(lenStr, NOESEEK_REL)# skip path

for one byte string length use lenStr = bs.readByte()
for DWORD (4 bytes) string length use lenStr = bs.readInt()

(not tested but should work)
Bigchillghost, Reverse Engineering a Game Model: viewtopic.php?f=29&t=17889
extracting simple models: viewtopic.php?f=29&t=10894
Make_H2O-ForzaHor3-jm9.zip
"You quoted the whole thing, what a mess."

dibe91
mega-veteran
mega-veteran
Posts: 195
Joined: Tue Jul 29, 2014 9:06 am
Has thanked: 3 times
Been thanked: 12 times

Re: Noesis tutorial Basic Model

Post by dibe91 » Wed Aug 01, 2018 10:41 am

I have a problem. I'm trying to create a script that can import dark souls 2 models by modifying an old dark souls script 1. if I try to import a dark souls 2 model of the basic version of the game, it works correctly:

Image

while if I try to import a model of dark souls 2 scholar of the first sin this happens to me:

Image

I leave you the complete script. I have only changed the type of model to be imported.

Code: Select all

'''Dark Souls 2 .flv model importer.

Note that there are little and big endian versions of the game, and they
are basically the same format. Therefore, all'''

from inc_noesis import *
import noesis
import rapi
import os

logd = open("darksouls2.log","w")

def registerNoesisTypes():
    '''Register the plugin. Just change the Game name and extension.'''
    
    handle = noesis.register("Dark Souls2", ".flv")
    noesis.setHandlerTypeCheck(handle, noepyCheckType)
    noesis.setHandlerLoadModel(handle, noepyLoadModel)
    return 1

def noepyCheckType(data):
    '''Verify that the format is supported by this plugin. Default yes'''
    
    if len(data) < 16:
        return 0
    try:
        bs = NoeBitStream(data)
        idstring = noeStrFromBytes(bs.readBytes(6))
        if idstring == "FLV":
            return 1
        return 0
    except:
        return 0
    
def get_endian(data):
    
    bs = NoeBitStream(data)
    bs.seek(6)
    endian = bs.readByte()
    if endian == 0x4C: # "L"
        return 0
    elif endian == 0x42: # "B"
        return 1 

def noepyLoadModel(data, mdlList):
    '''Build the model, set materials, bones, and animations. You do not
    need all of them as long as they are empty lists (they are by default)'''
    
    endian = get_endian(data)
    ctx = rapi.rpgCreateContext()
    parser = SanaeParser(data, endian)
    parser.parse_file()
    mdl = rapi.rpgConstructModel()
    mdl.setModelMaterials(NoeModelMaterials(parser.texList, parser.matList))
    mdl.setBones(parser.boneList)
    mdl.setAnims(parser.animList)
    mdlList.append(mdl)
    return 1

def logD(what):
    logFile = open("fmt_DarkSouls2_flv.log")

class FLV_mesh(object):
    
    def __init__(self):
        
        self.numFaceGroups = 0
        self.numIndices = []
        self.idxOffsets = []
        self.idxBuffs = [] #one mesh may have multiple parts
        self.numVerts = 0
        self.vertSize = 0
        self.vertOfs = 0
        self.vertSectionSize = 0
        self.vertBuff = bytes()
        self.uvBuff = bytes()
        self.matIds = 0

class FLV_mat(object):
    def __init__(self):
        self.Name1 = ""
        self.MTDName = ""
        self.texName = ""
        self.nParams = 0
        self.paramStartIndex = 0
        self.unknown0 = 0

class SanaeParser(object):
    
    def __init__(self, data, endian=0):
        
        self.inFile = NoeBitStream(data, endian)
        self.animList = []
        self.texList = []
        self.matList = []
        self.boneList = []
        self.meshList = []
        self.materialList = []
        self.dataOfs = 0 #offset to mesh data        
        if endian:
            rapi.rpgSetOption(noesis.RPGOPT_BIGENDIAN, 1)
        
    def build_meshes(self):
        
        rapi.rpgSetOption(noesis.RPGOPT_TRIWINDBACKWARD, 1)
        for mesh in self.meshList:
            
            print(mesh.vertSize)
            #rapi
            if mesh.vertSize == 28:
                rapi.rpgBindPositionBufferOfs(mesh.vertBuff, noesis.RPGEODATA_FLOAT, 28, 0)
                #noesis.doException("vertSize = 28, UVs not implemented...yet")
                rapi.rpgBindUV1BufferOfs(mesh.uvBuff,noesis.RPGEODATA_FLOAT,8,0)
            elif mesh.vertSize == 32:
                rapi.rpgBindPositionBufferOfs(mesh.vertBuff, noesis.RPGEODATA_FLOAT, 32, 0)
                #rapi.rpgBindNormalBufferOfs(mesh.vertBuff, noesis.RPGEODATA_FLOAT, 32, 20)
                rapi.rpgBindUV1BufferOfs(mesh.uvBuff,noesis.RPGEODATA_FLOAT,8,0)
                #rapi.rpgBindUV1BufferOfs(mesh.vertBuff, noesis.RPGEODATA_HALFFLOAT, 32, 16)
                #noesis.doException("vertSize = 32, UVs not implemented...yet")
            elif mesh.vertSize == 36:
                rapi.rpgBindPositionBufferOfs(mesh.vertBuff, noesis.RPGEODATA_FLOAT, 36, 0)
                rapi.rpgBindUV1BufferOfs(mesh.uvBuff,noesis.RPGEODATA_FLOAT,8,0)
                #noesis.doException("vertSize = 36, UVs not implemented...yet")
            elif mesh.vertSize == 40:
                rapi.rpgBindPositionBufferOfs(mesh.vertBuff, noesis.RPGEODATA_FLOAT, 40, 0)
                rapi.rpgBindUV1BufferOfs(mesh.uvBuff,noesis.RPGEODATA_FLOAT,8,0)
                
                #rapi.rpgBindUV1BufferOfs(mesh.vertBuff, noesis.RPGEODATA_HALFFLOAT,40,36)
            elif mesh.vertSize == 44:
                rapi.rpgBindPositionBufferOfs(mesh.vertBuff, noesis.RPGEODATA_FLOAT, 44, 0)
                rapi.rpgBindUV1BufferOfs(mesh.uvBuff,noesis.RPGEODATA_FLOAT,8,0)
                #noesis.doException("vertSize = 44, UVs not implemented...yet")
            
            #for j in range(mesh.numFaceGroups): # Not sure
            for j in range(1):
                numIdx = mesh.numIndices[j]
                idxBuff = mesh.idxBuffs[j]
                rapi.rpgSetMaterial(self.materialList[mesh.matIds].Name1)
                rapi.rpgCommitTriangles(idxBuff, noesis.RPGEODATA_USHORT, numIdx, noesis.RPGEO_TRIANGLE_STRIP, 1)
            #rapi.rpgCommitTriangles(None, noesis.RPGEODATA_USHORT, mesh.numVerts, noesis.RPGEO_POINTS, 1)
                
            
    def get_indices(self, numIdx):
        
        return self.inFile.readBytes(numIdx*2)

    def jnr(self,offs):
        oldOffs = self.inFile.tell()
        self.inFile.seek(offs)
        eos = False
        #strEnd = '00'.decode('hex')
        tmpBuffer = ""
        while not eos:
            tmpByte = self.inFile.readBytes(1)
            self.inFile.readBytes(1)
            tmpChar = ord(tmpByte)
            if(ord(tmpByte) == 0):
                eos = True
                break
            else:
                tmpBuffer += str(chr(ord(tmpByte)))
        self.inFile.seek(oldOffs)
        return tmpBuffer

    def jnr2(self,offs):
        oldOffs = self.inFile.tell()
        self.inFile.seek(offs)
        eos = False
        #strEnd = '00'.decode('hex')
        tmpBuffer = ""
        while not eos:
            tmpByte = self.inFile.readBytes(1)
            self.inFile.readBytes(1)
            tmpChar = ord(tmpByte)
            if(ord(tmpByte) == 0):
                eos = True
                break
            else:
                tmpBuffer += str(chr(ord(tmpByte)))
        newOffs = self.inFile.tell()
        self.inFile.seek(oldOffs)
        return [tmpBuffer,newOffs]

    def parse_faces(self):
        
        for mesh in self.meshList:
            for i in range(mesh.numFaceGroups):
                numIdx = mesh.numIndices[i]
                idxOfs = mesh.idxOffsets[i]
                self.inFile.seek(idxOfs)
                idxBuff = self.get_indices(numIdx)
                mesh.idxBuffs.append(idxBuff)
                
                
    def parse_vertices(self):
        
        for mesh in self.meshList:
            self.inFile.seek(mesh.vertOfs)
            #logd.write(str(mesh.vertSize) + " : " + str(mesh.vertOfs) + "," + str(mesh.vertSectionSize) + "\n")
            mesh.vertBuff = self.inFile.readBytes(mesh.vertSectionSize)
            #if(mesh.vertSize == 40):
            for i in range(mesh.numVerts):
                vS = mesh.vertSize
                vPos = i * vS
                tmpVert = mesh.vertBuff[vPos:vPos+mesh.vertSize]
                vU = struct.pack("f",float(struct.unpack("h",tmpVert[(vS-4):(vS-2)])[0] / 1024.0))
                vV = struct.pack("f",float(struct.unpack("h",tmpVert[(vS-2):(vS)])[0] / 1024.0))
                mesh.uvBuff += vU
                mesh.uvBuff += vV


    def resolveTextureName(self,inName):
        tmpName1 = inName.split('\\')
        logd.write(str(tmpName1[len(tmpName1)-1].split('.')))
        tmpName2 = tmpName1[len(tmpName1) - 1].split('.')
        type1 =  self.myWD + "\\" + self.myOutDIR + "\\" + tmpName2[0] + ".dds"
        type2 =  self.myWD + "\\" + self.myOutDIR + "\\" + tmpName2[0] + "_n [Unknown24].dds"
        logd.write("TYPE1: " + type1)
        #type1 = self.findInSubDir(tmpName2[0] + ".dds")
        #if(type1 == -1):
        #    return (tmpName2[0] + ".dds")
        #else:
        #    return type1
        return [type1,type2]
                        
                    

    def findInSubDir(self,filename,subdirectory = ''):
        logd.write("myWD = " + self.myWD + "\n")
        if subdirectory:
            path = subdirectory
        else:
            path = self.myWD
            for root, dirs, names in os.walk(path):
                if filename in names:
                    return os.path.join(root,filename)
                else:
                    return -1





    def parse_materials(self, numMat):
        for m in range(numMat):# in self.materialList:
            logd.write("PARSE MATERIAL INFO: " + str(self.inFile.tell()) + "\n")
            Name1 = self.jnr(self.inFile.readUInt())
            tmpMatThing = self.jnr2(self.inFile.readUInt())
            MTDName = tmpMatThing[0]
            tmpTexz = self.resolveTextureName(self.jnr(tmpMatThing[1]))
            texName = tmpTexz[0]
            normalName = tmpTexz[1]
            nParams = self.inFile.readUInt()
            paramStartIndex = self.inFile.readUInt()
            unknown0 = self.inFile.readUInt()
            self.inFile.read('3L')
            #logd.write("MATERIAL {}: {}\n".format(mat.Name1,mat.texName))
            mat = FLV_mat()
            mat.Name1 = Name1
            self.materialList.append(mat)
            
            material = NoeMaterial(Name1,texName)
            material.setNormalTexture(normalName)
            self.matList.append(material)

            
    def parse_bones(self, numBones):
        
        for i in range(numBones):
            self.inFile.seek(64, 1)
            
    def parse_unk1(self, count):
        
        for i in range(count):
            #logd.write("unk1: " + str(self.inFile.tell()))
            self.inFile.seek(128, 1)
            
    def parse_part_info(self, numParts):
        
        for mesh in self.meshList:
            #logd.write("part_info: " + str(self.inFile.tell()) + "\n")
            logd.write("PARSE PART INFO: " + str(self.inFile.tell()) + "\n")
            self.inFile.read('1L')
            matIds = self.inFile.readUInt()
            self.inFile.read('6L')
            numFaceGroups = self.inFile.readUInt()
            self.inFile.read('3L')
            #logd.write("NUM FACESGROUPS:" + str(numFaceGroups) + "\n")
            mesh.numFaceGroups = numFaceGroups
            mesh.matIds = matIds
            
    def parse_face_info(self):
        
        for mesh in self.meshList:
            for i in range(mesh.numFaceGroups):
                groupNum = self.inFile.readUInt()
                self.inFile.readUInt()
                numIdx = self.inFile.readUInt()
                #logd.write("NUM FACES:" + str(numIdx) + "\n")
                idxOfs = self.inFile.readUInt() + self.dataOfs
                idxSize = self.inFile.readUInt()
                self.inFile.read('3L')
                
                mesh.numIndices.append(numIdx)
                mesh.idxOffsets.append(idxOfs)
                            
    def parse_vertex_info(self):
        
        for mesh in self.meshList:
            self.inFile.read('1L')
            vertDescriptor = self.inFile.read('1L')
            vertSize = self.inFile.readUInt()
            numVerts = self.inFile.readUInt()
            self.inFile.readUInt()
            unk = self.inFile.readUInt()
            sectionSize = self.inFile.readUInt()
            vertOfs = self.inFile.readUInt() + self.dataOfs
            
            mesh.numVerts = numVerts
            mesh.vertSize = vertSize
            mesh.vertOfs = vertOfs
            mesh.vertSectionSize = sectionSize
            
    def getfdir(self):
        tmpDir = rapi.getInputName().split('\\')
        myCWD = ''
        for i in range(len(tmpDir) - 1):

            if(i == (len(tmpDir) - 2)):
                myCWD += tmpDir[i]
            else:
               myCWD += tmpDir[i] + '\\'
        
        return [myCWD,tmpDir[len(tmpDir) - 1].split('.')[0]]



    def parse_file(self):
        '''Main parser method'''
        tmpTing = self.getfdir()
        self.myWD = tmpTing[0]
        self.myOutDIR = tmpTing[1] +"_unpack"
        logd.write("myOutDIR = " + self.myOutDIR + "\n")
        
        

        #logd.write(rapi.getInputName())
        #header
        idstring = self.inFile.readBytes(6)
        
        #version?
        unk, type1, type2 = self.inFile.read('3H')
        
        self.dataOfs = self.inFile.readUInt()
        dataSize = self.inFile.readUInt()
        numBones = self.inFile.readUInt()
        numMat = self.inFile.readUInt()
        count = self.inFile.readUInt()
        numParts = self.inFile.readUInt()
        numMesh = self.inFile.readUInt()
        
        #create some mesh objects
        for i in range(numMesh):
            mesh = FLV_mesh()
            self.meshList.append(mesh)

        #for i in range(numMat):
        #    mat = FLV_mat()
        #    self.materialList.append(mat)
        
        self.inFile.read('6f')
        self.inFile.seek(64, 1)
        self.parse_bones(numBones)
        self.parse_materials(numMat)
        self.parse_unk1(count)
        self.parse_part_info(numParts)
        self.parse_face_info()        
        logd.write("PARSE VERTEX INFO: " + str(self.inFile.tell()) + "\n")
        self.parse_vertex_info()
        logd.write("PARSE FACES: " + str(self.inFile.tell()) + "\n")
        #parse data
        self.parse_faces()
        logd.write("PARSE VERTS: " + str(self.inFile.tell()) + "\n")
        self.parse_vertices()
        logd.write("END FILE ACCESS? " + str(self.inFile.tell()) + "\n")
        self.build_meshes()

tainhx
beginner
Posts: 33
Joined: Tue Jun 26, 2018 10:03 am
Has thanked: 2 times

Re: Noesis tutorial Basic Model

Post by tainhx » Sat Jan 05, 2019 8:20 am

Need tutorial for import Bone and Animation :wink:

User avatar
shakotay2
MEGAVETERAN
MEGAVETERAN
Posts: 2612
Joined: Fri Apr 20, 2012 9:24 am
Location: Nexus, searching for Jim Kirk
Has thanked: 642 times
Been thanked: 1339 times

Re: Noesis tutorial Basic Model

Post by shakotay2 » Fri Jan 18, 2019 10:26 pm

For skeleton bones look here:
https://forum.xentax.com/viewtopic.php?f=29&t=19429

You may read here, too:
viewtopic.php?f=13&t=14446&hilit=NoeKeyFramedValue

Be sure to read this documentation located in Noesis/Plugins/Python/__NPReadMe.txt
as advised by Gh0stBlade.
Bigchillghost, Reverse Engineering a Game Model: viewtopic.php?f=29&t=17889
extracting simple models: viewtopic.php?f=29&t=10894
Make_H2O-ForzaHor3-jm9.zip
"You quoted the whole thing, what a mess."

Post Reply