XeNTaX Forum Index
Forum MultiEx Commander Tools Tools Home
It is currently Fri Aug 17, 2018 9:51 pm

All times are UTC + 1 hour


Forum rules


Please click here to view the forum rules



Post new topic Reply to topic  [ 22 posts ]  Go to page Previous  1, 2
Author Message
 Post subject: Re: NexusTK .epf Tile Palette
PostPosted: Sat Oct 07, 2017 1:31 pm 
Offline
VVIP member
VVIP member
User avatar

Joined: Wed Jun 30, 2004 3:01 pm
Posts: 536
Location: Australia
Has thanked: 0 time
Have thanks: 8 times

Hi Dizzy,

Not sure if you need any of this information for your tool, or even if you want your tool to work with other images in the game, but here is some additional information about the following archives...
arrow, body, bow, coat, emotion, face, facedec, fan, hair, hairdec, helmet, mantle, neck, shield, shoes, spear, sword

These archives all have 2 corresponding entries in the file char.dat with the extensions PAL and DSC. The PAL one is the format you describe on your GitHub, but the DSC one is different. It kinda takes the place of the TBL file used by the tiles.

This is the DSC format, to the best of my knowledge so far...

Code:
  15 - Header (PartDescription)
  7 - null
  1 - Unknown (1)
  4 - Number of Parts
 
  // for each part (610 bytes per part)
    4 - Part ID (incremental from 0)
    4 - Palette ID Number
    4 - First Tile Number for this Part
    4 - Number of Tiles for this Part
    1 - Unknown (2)
    4 - Unknown
    1 - Unknown (4)
    4 - Unknown
    4 - Unknown
    4 - Number of Chunks (12)
 
    // for each chunk
      4 - Unknown ID
      4 - Unknown (-1/0)
      4 - Number of Blocks (4x4, then 4x2, then 4x6, to make the total of 12 blocks)
   
      // for each block
        1 - Unknown ID
        4 - null
        4 - Unknown (-1)


On order to read the images from the EPF files and map them to the correct PAL palette, you need to use the following fields above...
Code:
    4 - Palette ID Number
    4 - First Tile Number for this Part
    4 - Number of Tiles for this Part


For each Part, you can read the 3 fields above. The first one gives the palette ID for all images in the Part. The "First Tile Number" is the ID of the first image in the EPF file that belongs to this Part, and the "Number of Tiles" tells you the number of images in the Part.

For an example, the first entry in the DSC file might have "First Tile Number" = 0 and "Number of Tiles" = 20. In this case, all the images from 0-19 in the EPF file belong to this Part, and all use the same Palette. The second DSC entry might have "First Tile Number" = 20 and "Number of Tiles" = 8. In this case, images 20-27 belong to the Part.

The EPF file is basically the same as you wrote on GitHub, but I needed to work out the width of the image data in the EPF file as "Width = Width + (0-X Position)" and similarly for the height of the image data.

As for the Chunks and the Blocks - I have no idea what they represent, and I haven't really looked further in to them, as it isn't needed for reading the images. That said, I assume it's either something to do with the animation of the images, or placement of the images relative to other images, something like that I guess.

Hope this helps.

You can make the ads go away by registering


_________________
Game Extractor - Read and write thousands of game archives!


Top
 Profile  
 
 Post subject: Re: NexusTK .epf Tile Palette
PostPosted: Mon Oct 09, 2017 12:28 pm 
Offline
VVIP member
VVIP member
User avatar

Joined: Wed Jun 30, 2004 3:01 pm
Posts: 536
Location: Australia
Has thanked: 0 time
Have thanks: 8 times
And for the remaining archives...

Effects (efx.dat)...

Code:
// FRM FILE...
  4 - Number of Tiles
 
  // for each tile
    4 - Color Palette Number


Very simple. Seemed to be 40 images short of the number listed in the FRM file, but the images looked ok when I applied the palettes to them, so not sure what's going on there.


Monsters (mon.dat)...

Code:
// DNA FILE...
  4 - Number of Monsters
 
  // for each monster...
    4 - First Image ID
    1 - Number of Blocks
    1 - Unknown
    2 - Palette Number
   
    // for each Block
      2 - Number of Chunks
     
      // for each chunk
        2 - Unknown ID
        2 - Unknown ID
        2 - Unknown (-1)
        2 - null
        1 - null


Pretty sure the chunks represent the direction that the "monster" image is facing, and things like that. Haven't really explored in to it though.


For the EFX and MON, they both have EPF and PAL files as well, which are the same as all the others.

Hope this helps you, if you're so interested.

_________________
Game Extractor - Read and write thousands of game archives!


Top
 Profile  
 
 Post subject: Re: NexusTK .epf Tile Palette
PostPosted: Tue Oct 10, 2017 6:22 pm 
Offline
beginner

Joined: Wed Aug 30, 2017 1:20 am
Posts: 21
Has thanked: 4 times
Have thanks: 7 times
friendsofwatto, thank you for the information, I appreciate it!

The file_reader module is intended to handle all TK file formats, so these are great additions!

As for where/how to present all the different image formats/types in the QT5 GUI is another question. I'm open to suggestions on GUI/layout structure as this is not my forte :)


Top
 Profile  
 
 Post subject: Re: NexusTK .epf Tile Palette
PostPosted: Thu Feb 01, 2018 11:41 pm 
Offline
beginner

Joined: Wed Aug 30, 2017 1:20 am
Posts: 21
Has thanked: 4 times
Have thanks: 7 times
friendsofwatto,

I got more time to look into the other file types. I am trying to read a Shield:

Shield0.epf
Shield.dsc
Shield.pal

Shield.pal expands into multiple palettes, which I handle, but when I try to read the EPF similar to the tiles (w/TBL) I get back negative numbers.

Here is how I am reading the DSC file:

Code:
"""The DSC file handler representing *.dsc files."""
class DSCHandler(FileHandler):
    _PART_COUNT_POS = 23

    def __init__(self, *args):
        super(DSCHandler, self).__init__(*args)
        self.seek(self._PART_COUNT_POS)
        self.part_count = self.read('int')
        self.parts = []
        for i in range(self.part_count):
            part = {}
            part['id'] = self.read('int')
            part['palette_id'] = self.read('int')
            part['frame_index'] = self.read('int')
            part['frame_count'] = self.read('int')
            self.seek(14, whence=1)
            part['chunk_count'] = self.read('int')
            part['chunks'] = []
            for j in range(part['chunk_count']):
                chunk = {}
                chunk['id'] = self.read('int')
                chunk['unk'] = self.read('int')
                chunk['block_count'] = self.read('int')
                chunk['blocks'] = []
                for k in range(chunk['block_count']):
                    block = {}
                    block['id'] = self.read('byte')
                    self.seek(4, whence=1)
                    block['unk'] = self.read('int')
                    chunk['blocks'].append(block)
                part['chunks'].append(chunk)
            self.parts.append(part)


I then wrote this script to try to save the first 20 frames of the first part:

Code:
#!/usr/bin/env python3

import binascii
import io
import os
import signal
import sys

from file_reader import DSCHandler
from file_reader import EPFHandler
from file_reader import PALHandler
from file_reader import MAPHandler
from file_reader import SObjTBLHandler
from file_reader import TBLHandler

from PIL import Image

_VERBOSE = False

pal_handler = PALHandler('./Shield.pal')
print('PAL Count: {}'.format(pal_handler.pal_count))
dsc_handler = DSCHandler('./Shield.dsc')
print('Part Count: {}'.format(dsc_handler.part_count))
epf_handler = EPFHandler('./Shield0.epf', pals=pal_handler.pals, dsc=dsc_handler)
for i in range(1):
    print('  [{}] Part ID: {}'.format(i+1, dsc_handler.parts[i]['id']))
    print('  [{}] Palette ID: {}'.format(i+1, dsc_handler.parts[i]['palette_id']))
    print('  [{}] Frame Index: {}'.format(i+1, dsc_handler.parts[i]['frame_index']))
    print('  [{}] Frame Count: {}'.format(i+1, dsc_handler.parts[i]['frame_count']))
    if _VERBOSE:
        print('')
        print('  [{}] Chunk Count: {}'.format(i+1, dsc_handler.parts[i]['chunk_count']))
        for j in range(dsc_handler.parts[i]['chunk_count']):
            print('    [{}] Chunk ID: {}'.format(i+1, dsc_handler.parts[i]['chunks'][j]['id']))
            print('    [{}] Unknown: {}'.format(i+1, dsc_handler.parts[i]['chunks'][j]['unk']))
            print('    [{}] Block Count: {}'.format(i+1, dsc_handler.parts[i]['chunks'][j]['block_count']))
            for k in range(dsc_handler.parts[i]['chunks'][j]['block_count']):
                print('      [{}] Block ID: {}'.format(i+1, dsc_handler.parts[i]['chunks'][j]['blocks'][k]['id']))
                print('      [{}] Unknown: {}'.format(i+1, dsc_handler.parts[i]['chunks'][j]['blocks'][k]['unk']))


    alpha_rgb=(0, 0, 0)
    background_color='black'
   
    for i in range(dsc_handler.parts[0]['frame_count']):
        p1_first_frame = dsc_handler.parts[0]['frame_index']
        frame = epf_handler.frames[p1_first_frame+i]
        width = frame['width']
        height = frame['height']

        pixel_data_offset = frame['pixel_data_offset']
        stencil_data_offset = frame['stencil_data_offset']

        pixel_data = epf_handler.pixel_data[pixel_data_offset:pixel_data_offset+(width*height)]

        if not epf_handler.dsc:
            palette_index = 0
        else:
            palette_index = epf_handler.dsc.parts[0]['palette_id']

        pixel_bytes = []
        for j in range(height * width):
            pixel_byte = (epf_handler.pals[palette_index]['colors'][pixel_data[j]]['rgb'])
            if pixel_byte in epf_handler._ALPHA_COLORS:
                pixel_byte = alpha_rgb

            pixel_bytes.append(pixel_byte)

        if frame['top'] or frame['left'] or (height != 24 or width != 24):
            sub_image = Image.new('RGBA', (width, height), background_color)
            sub_image.putdata(pixel_bytes)

            image = Image.new('RGBA', (24, 24), background_color)
            image.paste(sub_image, (frame['left'], frame['top']))
        else:
            image = Image.new('RGBA', (width, height))
            image.putdata(pixel_bytes)

        image.save('C:\\Shield\\frame{}.bmp'.format(i))


I get the following error when I try to run it:
Code:
PAL Count: 22
Part Count: 56
  [1] Part ID: 0
  [1] Palette ID: 0
  [1] Frame Index: 0
  [1] Frame Count: 20
Traceback (most recent call last):
  File "C:\Users\Stephen\Git\TKViewer\test.py", line 70, in <module>
    sub_image = Image.new('RGBA', (width, height), background_color)
  File "C:\Users\Stephen\AppData\Local\Programs\Python\Python36-32\lib\site-packages\PIL\Image.py", line 2282, in new
    _check_size(size)
  File "C:\Users\Stephen\AppData\Local\Programs\Python\Python36-32\lib\site-packages\PIL\Image.py", line 2261, in _check_size
    raise ValueError("Width and height must be >= 0")
ValueError: Width and height must be >= 0


Here is my File structure description:

Code:
byte[15] header              (15 bytes) # PartDescription
byte[7] null                 (7 bytes)
byte unknown1                (1 byte)
int part_count               (4 bytes)
part[part_count] parts       (27 + (part_count * part_size) bytes)

typedef struct {
  int id                     (4 bytes)
  int palette_id             (4 bytes)
  int frame_index            (4 bytes)
  int frame_count            (4 bytes)
  byte unknown2              (1 byte)
  int unknown3               (4 bytes)
  byte unknown4              (1 byte)
  int unknown5               (4 bytes)
  int unknown6               (4 bytes)
  int chunk_count            (4 bytes)
  chunk[chunk_count] chunks  (34 + (chunk_count * chunk_size) bytes)
} part

typedef struct {
  int id                     (4 bytes)
  int unknown                (4 bytes)
  int block_count            (4 bytes)
  block[block_count] blocks  (16 + (block_count * block_size) bytes)
} chunk

typedef struct {
  byte id                    (1 byte)
  int null                   (4 bytes)
  int unknown                (4 bytes)
} block                      (9 bytes)


Anything wrong with my interpretation of your DSC post?

Thanks!


Top
 Profile  
 
 Post subject: Re: NexusTK .epf Tile Palette
PostPosted: Fri Feb 02, 2018 3:54 am 
Offline
beginner

Joined: Wed Aug 30, 2017 1:20 am
Posts: 21
Has thanked: 4 times
Have thanks: 7 times
I think the blocks are where I'm misreading:

Code:
4 - Number of Blocks (4x4, then 4x2, then 4x6, to make the total of 12 blocks)


I started working out the file and the blocks stopped making sense:

Code:
# 50 61 72 74 44 65 73 63 72 69 70 74 69 6F 6E | (15) PartDescription
# 00 00 00 00 00 00 00 01                      | (08) Null[7], Unknown[1]
# 38 00 00 00                                  | (04) Part Count (56)
# 00 00 00 00                                  | (04)   1st Part ID
# 00 00 00 00                                  | (04)   1st Palette ID
# 00 00 00 00                                  | (04)   1st Frame Index
# 14 00 00 00                                  | (04)   1st Frame Count (20)
# 02 00 00 00 00 04 00 00 00 00 00 00 00 00    | (14)   Unknown[1,4,1,4,4]
# 14 00 00 00                                  | (04)   Chunk Count (20)
# 00 00 00 00                                  | (04)     1st Chunk ID
# FF FF FF FF                                  | (04)     1st Chunk Unknown
# 04 00 00 00                                  | (04)     Block Count (4)
# ================================================================================#
# 00                                           | (01)       1st Block ID
# 00 00 00 00                                  | (04)       1st Block Null
# FF FF FF FF                                  | (04)       1st Block Unknown
# ================================================================================#
# 01                                           | (01)       2nd Block ID
# 00 00 00 00                                  | (04)       2nd Block Null
# FF FF FF FF                                  | (04)       2nd Block Unknown
# ================================================================================#
# 00                                           | (01)       3rd Block ID
# 00 00 00 00                                  | (04)       3rd Block Null
# FF FF FF FF                                  | (04)       3rd Block Unknown
# ================================================================================#
# 02                                           | (01)       4th Block ID
# 00 00 00 00                                  | (04)       4th Block Null
# FF FF FF FF                                  | (04)       4th Block Unknown
# ================================================================================#
# 01                                           | (01)       ?th Block ID
# 00 00 00 00                                  | (04)       ?th Block Null
# FF FF FF FF                                  | (04)       ?th Block Unknown
# ================================================================================#
# 04                                           | (01)       ?th Block ID
# 00 00 00 00                                  | (04)       ?th Block Null
# FF FF FF FF                                  | (04)       ?th Block Unknown
# ================================================================================#
# 00 00 00 03
# 00 00 00 00
# FF FF FF FF
# 04
# 00 00 00 00
# FF FF FF FF


I'm sure it's due to the block offset being wrong


Top
 Profile  
 
 Post subject: Re: NexusTK .epf Tile Palette
PostPosted: Sat Feb 03, 2018 10:14 pm 
Offline
beginner

Joined: Wed Aug 30, 2017 1:20 am
Posts: 21
Has thanked: 4 times
Have thanks: 7 times
Updating this a bit with some calculations:
Code:
Length of DSC File: 0xE94B
Length of Header: 0x1B
Number of Parts: 0x38

(0xE94B - 0x1B) / 0x38 = 0x42A || 1066 bytes


Part Size is 0x42A (1066) bytes

I could parse the file by part boundary and ignore chunk/block data, but I'd like to figure out how to properly read it


Top
 Profile  
 
 Post subject: Re: NexusTK .epf Tile Palette
PostPosted: Tue Feb 06, 2018 1:54 am 
Offline
beginner

Joined: Wed Aug 30, 2017 1:20 am
Posts: 21
Has thanked: 4 times
Have thanks: 7 times
Figured out my issue.

The issue was that in the EPF File's frames, the top, right, bottom, left - shorts are suppose to be signed, not unsigned.



Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 22 posts ]  Go to page Previous  1, 2

All times are UTC + 1 hour


Who is online

Users browsing this forum: Anexenaumoon and 4 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group