XeNTaX Forum Index
Forum MultiEx Commander Tools Tools Home
It is currently Fri Nov 16, 2018 4:52 am

All times are UTC + 1 hour


Forum rules


Please click here to view the forum rules



Post new topic Reply to topic  [ 14 posts ] 
Author Message
 Post subject: MECC Opening Night Bitmap Files
PostPosted: Mon Nov 05, 2018 12:58 am 
Offline
beginner

Joined: Wed Sep 23, 2009 5:16 pm
Posts: 39
Has thanked: 0 time
Have thanks: 2 times
Hey Guys,

I've successfully written a small tool for extracting files from a game called "Opening Night" made by MECC in the 90's.

The compression on the bitmap files has me a bit stumped though, and I was wondering if it looked like anything you've encountered before.

I'm 99.9% sure this is data for an indexed color bitmap file cause there is a file called "COLRLST" I extracted that is a color table for 245 colors that the game uses. The "CBMP" files as they're called in the game archive consist of a stripped down BMP header, including sizeX, sizeY, etc, then the bitmap data itself. Here's a screen shot of the highlighted bitmap data that's confusing me. This particular file is 80 by 80 pixels when it's uncompressed.

At first I thought it could be RLE compression, but I don't see any 00's for line breaks and stuff. Anyone know what kind of compression this is?


Attachments:


You do not have the required permissions to view the files attached to this post. Register to gain access.


_________________
:twisted: jawharp :twisted:


Top
 Profile  
 
 Post subject: Re: MECC Opening Night Bitmap Files
PostPosted: Mon Nov 05, 2018 6:16 am 
Offline
beginner

Joined: Wed Sep 23, 2009 5:16 pm
Posts: 39
Has thanked: 0 time
Have thanks: 2 times
I started dicking around with changing some of the values in the file data then relaunching the program.

I first changed the second byte of bitmap data to "c9" (just a random value) and it appears to have changed a color in the image, not the pixel data. I think there might be additional color information at the beginning of these files after the header I initially posted about.

I then started changing every second byte of bitmap data to "c9" to see if more colors would be replaced and they were.

I posted some images here so you can see what I'm talking about.

Image
The 80x80 pixel icon I originally posted about.


Image
The byte I changed to "c9"

Image
The result.

Image
More bytes changed to "c9"

Image
The second result.

_________________
:twisted: jawharp :twisted:


Top
 Profile  
 
 Post subject: Re: MECC Opening Night Bitmap Files
PostPosted: Mon Nov 05, 2018 9:39 am 
Offline
veteran

Joined: Sat Oct 21, 2006 12:58 pm
Posts: 124
Has thanked: 5 times
Have thanks: 18 times
It would help if you post a few sample files somewhere along the colorlst file if you want somebody take a look, looking at mere hex code isn't always useful.

The data could be uncompressed for all we know as you've not provided any filesize infos either, alternatively presence of 00s is not mandatory for RLE executions (of which there are many), not all of them declare line ends etc.


Top
 Profile  
 
 Post subject: Re: MECC Opening Night Bitmap Files
PostPosted: Mon Nov 05, 2018 1:49 pm 
Offline
beginner

Joined: Wed Sep 23, 2009 5:16 pm
Posts: 39
Has thanked: 0 time
Have thanks: 2 times
Here is a link to some cbmp files and the colorlst file:

http://s000.tinyupload.com/index.php?fi ... 8712648431

This data is most likely compressed since 80 x 80 pixels = 6400 pixels worth of data, and the file is only 2380 bytes in length.

If anyone wants to take a look, any info you might have would be most appreciated. Here's what I have for the header info:

offset 00 - 13:
header containing # of images in the cbmp file, horizontal size, vertical size, length of this header, size of bmp data

offset 13 - 25:
second header containing length of second header, i *think* total # of colors, a 4 byte mystery number, then the total number of pixels in the final image.

offset 26 and on: bitmap data

_________________
:twisted: jawharp :twisted:


Top
 Profile  
 
 Post subject: Re: MECC Opening Night Bitmap Files
PostPosted: Sat Nov 10, 2018 12:11 pm 
Offline
veteran

Joined: Sat Oct 21, 2006 12:58 pm
Posts: 124
Has thanked: 5 times
Have thanks: 18 times
I looked at this a bit but there is odd problems, first of all, the colorlist does not match 100% at all vs the screenshot image content, there is noticeable quality deteriorating matching back to it which should not be the case or there is other colorlist that is used as I tried both RGB and BGR orders to be sure.

Other curious thing is the amount of colors chosen 245, one would expect 255, perhaps the remaining 10 were reserved for art specific colors or are used for file encoding method.

Now if we google up BMP compression spec: https://docs.microsoft.com/en-us/window ... ompression

and we can also test save one of those 80x80 into a bmp with rle indexed, we get 5492 bytes or so and also looking at the data vs spec, it definitely does not look like the BMP RLE method which can't achieve 2391 bytes filesize. There is the possibility of the huffman 1d encoding which is stated to be one method to encode for bmps but it just remains vague mention on the net and most apps just use the RLE method.

Curiously enough, pcx achieves 3917 bytes with it's RLE for comparison and again, it seems ever more likely it is not RLE and we are dealing with something like GIF's LZW compress but that is just a guess.

Would need to figure out which art is which file and have the screenshotted same art match properly into the colorlist so the file encoding can be compared and figured out easier, worst case scenario is reverse engineering the executable relevant code how it accesses those files.
As of now, it can't be told what the precise encoding method is used.


Top
 Profile  
 
 Post subject: Re: MECC Opening Night Bitmap Files
PostPosted: Sat Nov 10, 2018 10:49 pm 
Offline
beginner

Joined: Wed Sep 23, 2009 5:16 pm
Posts: 39
Has thanked: 0 time
Have thanks: 2 times
Sadly I figured as much. I've opened up the exe in OllyDbg and IDA, and I've definitely found the functions that are doing things with CBMP files, but I don't know how to read the assembly and the pseudocode isn't much help either for me.

Here are some files that are in a folder in the archive called "RLES." I'm pretty sure given the name these are RLE bitmaps. What I've made of the header so far is:

2 bytes: # of images in RLE file
2 bytes: Size X
2 bytes: Size Y
4 bytes: offset of pixel data
4 bytes: length of pixel data in bytes

All header values are big endian. There is no color information, so I'm assuming it's using the same COLORLST file as the CBMP files.

I'm attaching a zip of 4 of the RLE files. Do these look like regular RLE pixel data?


Attachments:


You do not have the required permissions to view the files attached to this post. Register to gain access.


_________________
:twisted: jawharp :twisted:


Top
 Profile  
 
 Post subject: Re: MECC Opening Night Bitmap Files
PostPosted: Sun Nov 11, 2018 11:31 am 
Offline
veteran

Joined: Sat Oct 21, 2006 12:58 pm
Posts: 124
Has thanked: 5 times
Have thanks: 18 times
jawharp wrote:
Here are some files that are in a folder in the archive called "RLES." I'm pretty sure given the name these are RLE bitmaps. What I've made of the header so far is:

2 bytes: # of images in RLE file
2 bytes: Size X
2 bytes: Size Y
4 bytes: offset of pixel data
4 bytes: length of pixel data in bytes

All header values are big endian. There is no color information, so I'm assuming it's using the same COLORLST file as the CBMP files.

I'm attaching a zip of 4 of the RLE files. Do these look like regular RLE pixel data?


Your header info checks out all good from quick look.

Yeap, look definitely RLE coding, I've looked up the pattern and it seems that for example with file 4016.RLES, the actual pixel data starts by 67,0 (0x43,0x00) where first byte is RLE count so how many times to repeat the next byte, thus 67 times of zero and zero here is possibly background transparent pixels (white is curious choice of background tho). Since full dimension is 76x91, it does make sense so far leaving us with 9 pixels from full width for first line and it does not seem there is any line end markers.

Now then when there is 0x80 (128) given for the RLE count field, it seems to be that you must copy following amount of bytes as is command thus 128=1 byte to copy,129=2 byte,130=3 bytes... and after that fall back to RLE parsing thus every value is RLE unless get 0x80+++ tell to copy as is instead. It should be noted that all data is RLEd, not just background zero pixels from my glance.

In any case, you'll have to code a tool and see if get something sensible from that as raw bytes and of course enough output bytes to fill the full dimension, else there may be still something overlooked.

Oh yes, those CBMP files could be 24-bit with unknown compression thus why color list doesn't match them properly as one explanation.


Top
 Profile  
 
 Post subject: Re: MECC Opening Night Bitmap Files
PostPosted: Sun Nov 11, 2018 3:12 pm 
Offline
beginner

Joined: Wed Sep 23, 2009 5:16 pm
Posts: 39
Has thanked: 0 time
Have thanks: 2 times
Thanks for the intel! I'll start on that tool.

As for the CBMP files being 24-bit, do you think they could still be 24-bit even though the game installer says the game requires 256 Color mode in Windows?

_________________
:twisted: jawharp :twisted:


Top
 Profile  
 
 Post subject: Re: MECC Opening Night Bitmap Files
PostPosted: Tue Nov 13, 2018 10:57 am 
Offline
veteran

Joined: Sat Oct 21, 2006 12:58 pm
Posts: 124
Has thanked: 5 times
Have thanks: 18 times
jawharp wrote:
Thanks for the intel! I'll start on that tool.

As for the CBMP files being 24-bit, do you think they could still be 24-bit even though the game installer says the game requires 256 Color mode in Windows?


It's just a one theory for the color mismatch issue, other scenario is that they are also 256 color but there are other color lists matching them (or the missing additional 10 colors) or color list is within under compression or there is look up table that brightens etc the color list colors when app is opened throwing it off but thats not likely.

It would help if could determine what content visually specific CBMP holds by potentially switching them around but given game uses archives this would be tricky at best to swap in them just to get better clue on uncompressed size vs the compressed.

See the png for example, I loaded the colors on it and many are changed if compare to the screenshot original in closer zoom.


Attachments:


You do not have the required permissions to view the files attached to this post. Register to gain access.



Top
 Profile  
 
 Post subject: Re: MECC Opening Night Bitmap Files
PostPosted: Tue Nov 13, 2018 8:51 pm 
Offline
beginner

Joined: Wed Sep 23, 2009 5:16 pm
Posts: 39
Has thanked: 0 time
Have thanks: 2 times
So, I wrote a utility to decompress the RLE data and replace the pixels with the entries from the colorlst file. This is the most progress I've made in a while. I used 7yuv to view the raw pixel data generated from my tool, and once I learned that the x and y sizes are reversed in the RLES files, I was greeted with this:

Image

It's an asset in the game for a unicorn prop. Here's a shot of it in the game:

Image

I can't figure out why the colors are all smearing like that. When I try to add this data into a bmp file, windows says the file is corrupted.

Not sure what's going on with the colors, but I would call this progress.

_________________
:twisted: jawharp :twisted:


Top
 Profile  
 
 Post subject: Re: MECC Opening Night Bitmap Files
PostPosted: Tue Nov 13, 2018 9:01 pm 
Offline
beginner

Joined: Wed Sep 23, 2009 5:16 pm
Posts: 39
Has thanked: 0 time
Have thanks: 2 times
Here is the generated pixel color data file. It is 91 pixels wide by 76 pixels tall.


Attachments:


You do not have the required permissions to view the files attached to this post. Register to gain access.


_________________
:twisted: jawharp :twisted:


Top
 Profile  
 
 Post subject: Re: MECC Opening Night Bitmap Files
PostPosted: Tue Nov 13, 2018 9:33 pm 
Offline
beginner

Joined: Wed Sep 23, 2009 5:16 pm
Posts: 39
Has thanked: 0 time
Have thanks: 2 times
Here's another one.

Image


original in game:


Image


Attachments:


You do not have the required permissions to view the files attached to this post. Register to gain access.


_________________
:twisted: jawharp :twisted:


Top
 Profile  
 
 Post subject: Re: MECC Opening Night Bitmap Files
PostPosted: Wed Nov 14, 2018 12:10 am 
Offline
beginner

Joined: Wed Sep 23, 2009 5:16 pm
Posts: 39
Has thanked: 0 time
Have thanks: 2 times
Disregard last 2 posts. My decompress function had an error.

It worked!

RLE files handled. Going to write up something that exports the bitmap files.


Attachments:


You do not have the required permissions to view the files attached to this post. Register to gain access.


_________________
:twisted: jawharp :twisted:


Top
 Profile  
 
 Post subject: Re: MECC Opening Night Bitmap Files
PostPosted: Wed Nov 14, 2018 6:56 am 
Offline
veteran

Joined: Sat Oct 21, 2006 12:58 pm
Posts: 124
Has thanked: 5 times
Have thanks: 18 times
Looks like my RLE hunch was pretty close since got results, however I realised there is likely a background vs content issue apparently, the zero used as white inside the dog is not supposed to be 'hole' yet the dog outer edges of same zeroes is, it may be that the rle count byte zeros (1-127,0) should be changed to something else in RGB to separate the background (had this issue in other game too...) while 128+ ones kept as white, if this fixes the issue, else you will have to hand paint the edges for any of your own project use as I doubt your just decoding just to view the arts.

You got 4002.rle original to add here? would be useful to check the theory a bit


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 14 posts ] 

All times are UTC + 1 hour


Who is online

Users browsing this forum: No registered users and 2 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group