XeNTaX Forum Index
Forum MultiEx Commander Tools Tools Home
It is currently Sat May 27, 2017 2:40 pm

All times are UTC + 1 hour


Forum rules


Please click here to view the forum rules



Post new topic Reply to topic  [ 19 posts ]  Go to page 1, 2  Next
Author Message
 Post subject: Away3D decompression
PostPosted: Tue Jul 29, 2014 8:56 pm 
Offline
ultra-veteran
ultra-veteran
User avatar

Joined: Thu Feb 03, 2011 11:18 am
Posts: 476
Has thanked: 42 times
Have thanks: 287 times














You can make the ads go away by registering

I've been looking into Away3D models from Honda 3D Design Archives. The .awd files are compressed, and this format seems to use zlib or lzma.
I'm attaching a quickbms script I made for decompression.

The same engine is used by Nissan for their personalization pages - Juke and other models. Only this time the awd files are embedded in swf along with what appear to be compressed ATF textures.
Unfortunately I haven't been able to extract these textures; I can't even open them with ATFViewer from Adobe's Gaming SDK.

Update: For now I made a conversion script for quickbms.
Collection of scripts (sorry, I can only attach one file here)
- awd decompressor
- atf convertor
- abg decompressor
- abg to obj decompressor & converter
- maxscript for loading GarageFuse scenes


Attachments:


You do not have the required permissions to view the files attached to this post. Register to gain access.


_________________
Please post any requests or issues with my tools in the appropriate topics.
I'm sorry if I don't reply or if I ignore PMs. My time is very limited.


Last edited by Chipicao on Sat Aug 02, 2014 10:34 am, edited 2 times in total.

Top
 Profile  
 
 Post subject: Re: Away3D decompression
PostPosted: Wed Jul 30, 2014 9:57 am 
Offline
ultra-veteran
ultra-veteran
User avatar

Joined: Tue Jan 04, 2005 9:47 pm
Posts: 585
Has thanked: 43 times
Have thanks: 112 times
[out]


Last edited by Wobble on Sun Mar 12, 2017 12:04 pm, edited 1 time in total.

Top
 Profile  
 
 Post subject: Re: Away3D decompression
PostPosted: Wed Jul 30, 2014 10:29 am 
Offline
ultra-veteran
ultra-veteran
User avatar

Joined: Thu Feb 03, 2011 11:18 am
Posts: 476
Has thanked: 42 times
Have thanks: 287 times
Damn, that does sound awful.

Any idea where I could find the source code for this old encoder?
Is it by any chance the one on github? https://github.com/adobe/dds2atf It seems somewhat old, from 2 years ago.

_________________
Please post any requests or issues with my tools in the appropriate topics.
I'm sorry if I don't reply or if I ignore PMs. My time is very limited.


Top
 Profile  
 
 Post subject: Re: Away3D decompression
PostPosted: Wed Jul 30, 2014 12:24 pm 
Offline
ultra-veteran
ultra-veteran
User avatar

Joined: Tue Jan 04, 2005 9:47 pm
Posts: 585
Has thanked: 43 times
Have thanks: 112 times
[out]


Last edited by Wobble on Sun Mar 12, 2017 12:04 pm, edited 1 time in total.

Top
 Profile  
 
 Post subject: Re: Away3D decompression
PostPosted: Wed Jul 30, 2014 12:37 pm 
Offline
ultra-veteran
ultra-veteran
User avatar

Joined: Thu Feb 03, 2011 11:18 am
Posts: 476
Has thanked: 42 times
Have thanks: 287 times
Unfortuantely that doesn't match the samples I uploaded. Some would have format 4 or 5, others wold have height = 0.

To me it looks like everything after "ATF" is compressed, header included.

_________________
Please post any requests or issues with my tools in the appropriate topics.
I'm sorry if I don't reply or if I ignore PMs. My time is very limited.


Top
 Profile  
 
 Post subject: Re: Away3D decompression
PostPosted: Wed Jul 30, 2014 1:01 pm 
Offline
ultra-veteran
ultra-veteran
User avatar

Joined: Tue Jan 04, 2005 9:47 pm
Posts: 585
Has thanked: 43 times
Have thanks: 112 times
[out]


Last edited by Wobble on Sun Mar 12, 2017 12:04 pm, edited 1 time in total.

Top
 Profile  
 
 Post subject: Re: Away3D decompression
PostPosted: Wed Jul 30, 2014 2:01 pm 
Offline
ultra-veteran
ultra-veteran
User avatar

Joined: Tue Jan 04, 2005 9:47 pm
Posts: 585
Has thanked: 43 times
Have thanks: 112 times
[out]


Last edited by Wobble on Sun Mar 12, 2017 12:04 pm, edited 1 time in total.

Top
 Profile  
 
 Post subject: Re: Away3D decompression
PostPosted: Wed Jul 30, 2014 4:02 pm 
Offline
ultra-veteran
ultra-veteran
User avatar

Joined: Thu Feb 03, 2011 11:18 am
Posts: 476
Has thanked: 42 times
Have thanks: 287 times
Thanks, that really helped! It turns out only the first image (mip) is compressed, the rest are in Jpeg XR format like you said.
The structure is something like this (big endian):
Code:
// ATF format:  (OLD FORMAT!)
//
// U8[3] - signature - 'ATF'
// U24 - total file size
// U8 - format - 0=RGB888, 1=RGBA8888 (straight alpha), 2=Compressed(dxt1+pvrtc+etc1)
// U8 - width - texture size (2^n) (max n=11)
// U8 - height - texture size (2^n) (max n=11)
// U8 - count - total texture count (main + mip maps, max n=12)
// U24 - main texture size (compressed)
// lzma stream
// U24 - mip_2 size (uncompressed)
// mip_2 stream
// ...
// U24 - mip_n size
// mip_n stream


Binary 19 - mip 2:
Image

I'm now trying to extract the first texture, assuming it's also JXR, but I can't seem to get it working.
quickbms test script:
Code:
idstring "ATF"
endian big
get NAME basename
string NAME += "_main.jpg"
get FSIZE threebyte
get FORMAT byte
get WIDTH byte
get HEIGHT byte
get COUNT byte

comtype lzma
get ZSIZE threebyte
savepos OFFSET
clog NAME OFFSET ZSIZE ZSIZE


Error: the compressed LZMA input is wrong or incomplete (2)
stream was not finished

Error: there is an error with the decompression
the returned output size is negative (-102)

_________________
Please post any requests or issues with my tools in the appropriate topics.
I'm sorry if I don't reply or if I ignore PMs. My time is very limited.


Top
 Profile  
 
 Post subject: Re: Away3D decompression
PostPosted: Wed Jul 30, 2014 5:28 pm 
Offline
ultra-veteran
ultra-veteran
User avatar

Joined: Tue Jan 04, 2005 9:47 pm
Posts: 585
Has thanked: 43 times
Have thanks: 112 times
[out]


Last edited by Wobble on Sun Mar 12, 2017 12:04 pm, edited 1 time in total.

Top
 Profile  
 
 Post subject: Re: Away3D decompression
PostPosted: Wed Jul 30, 2014 5:35 pm 
Offline
ultra-veteran
ultra-veteran
User avatar

Joined: Thu Feb 03, 2011 11:18 am
Posts: 476
Has thanked: 42 times
Have thanks: 287 times
Ah, thanks, I forgot to look into atf.h.
That's really f*ed up, but I won't give up yet, first I want to see that stream unpacked.

I was under the impression that quickbms doesn't need the uncompressed size for lzma.

Edit: never mind, I used lzma_dynamic instead and it worked.

_________________
Please post any requests or issues with my tools in the appropriate topics.
I'm sorry if I don't reply or if I ignore PMs. My time is very limited.


Top
 Profile  
 
 Post subject: Re: Away3D decompression
PostPosted: Wed Jul 30, 2014 6:55 pm 
Offline
ultra-veteran
ultra-veteran
User avatar

Joined: Tue Jan 04, 2005 9:47 pm
Posts: 585
Has thanked: 43 times
Have thanks: 112 times
[out]


Last edited by Wobble on Sun Mar 12, 2017 12:05 pm, edited 2 times in total.

Top
 Profile  
 
 Post subject: Re: Away3D decompression
PostPosted: Wed Jul 30, 2014 9:55 pm 
Offline
ultra-veteran
ultra-veteran
User avatar

Joined: Thu Feb 03, 2011 11:18 am
Posts: 476
Has thanked: 42 times
Have thanks: 287 times
Thanks for the info, but for now I think I'm gonna settle with a quickbms converter from the old format to the new.
It turned out quite a bit complex because it's not just the main header that needs to be converted, but also every U24 length from every stream. Maybe that's why those files came up empty for you?

Code:
idstring "ATF"
endian big
get NAME basename
string NAME += "_v2.atf"
get FSIZE threebyte

if FSIZE == 0 #double check if new ATF format
   print "ATF file is already up to date"
   cleanexit
endif

get FORMAT byte
get WIDTH byte
get HEIGHT byte
get COUNT byte

if FORMAT == 0
   set SUBSTREAMS 1
elif FORMAT == 1
   set SUBSTREAMS 1
elif FORMAT == 2
   set SUBSTREAMS 8
endif

log MEMORY_FILE 0 0
putct "ATF" string -1 MEMORY_FILE
put 0xFF long MEMORY_FILE #reserved
put 0x2 byte MEMORY_FILE #version
Xmath FSIZE "FSIZE + (COUNT * SUBSTREAMS)"
put FSIZE long MEMORY_FILE
put FORMAT byte MEMORY_FILE
put WIDTH byte MEMORY_FILE
put HEIGHT byte MEMORY_FILE
put COUNT byte MEMORY_FILE

set MOFFSET 16
append
for i = 0 < COUNT
   for j = 0 < SUBSTREAMS
      get SIZE threebyte
      PutVarChr MEMORY_FILE MOFFSET SIZE long
      savepos OFFSET
      log MEMORY_FILE OFFSET SIZE
      goto SIZE 0 SEEK_CUR
      math MOFFSET += 4
      math MOFFSET += SIZE
   next j
next i
append

math FSIZE += 12
log NAME 0 FSIZE MEMORY_FILE

It should work with types 0 and 1 as well, but I don't have any to test. Too bad this ATFviewer doesn't have a save/export function.

Image

_________________
Please post any requests or issues with my tools in the appropriate topics.
I'm sorry if I don't reply or if I ignore PMs. My time is very limited.


Top
 Profile  
 
 Post subject: Re: Away3D decompression
PostPosted: Thu Jul 31, 2014 5:57 pm 
Offline
ultra-veteran
ultra-veteran
User avatar

Joined: Tue Jan 04, 2005 9:47 pm
Posts: 585
Has thanked: 43 times
Have thanks: 112 times
[out]


Last edited by Wobble on Sun Mar 12, 2017 12:05 pm, edited 1 time in total.

Top
 Profile  
 
 Post subject: Re: Away3D decompression
PostPosted: Thu Jul 31, 2014 6:22 pm 
Offline
ultra-veteran
ultra-veteran
User avatar

Joined: Thu Feb 03, 2011 11:18 am
Posts: 476
Has thanked: 42 times
Have thanks: 287 times
I'm gonna leave textures on hold for now, as I've found yet another container and 3D format used by Away3D.
The files bear a .abg extension and can be found in models from GarageFuse.com.

Here's a quickbms script to unpack them
Code:
get LEN short
getdstring VER LEN
get UNK long
get NAME basename
string name += "_unpacked.abg"
savepos OFFSET
get ZSIZE asize
math ZSIZE -= 6
math ZSIZE -= LEN

comtype lzma86head
clog NAME OFFSET ZSIZE ZSIZE


Inside there's a mix of plain-text and binary data, and the format looks similar to OBJ. I hope to have an importing maxscript ready soon.

_________________
Please post any requests or issues with my tools in the appropriate topics.
I'm sorry if I don't reply or if I ignore PMs. My time is very limited.


Last edited by Chipicao on Thu Jul 31, 2014 8:12 pm, edited 1 time in total.

Top
 Profile  
 
 Post subject: Re: Away3D decompression
PostPosted: Thu Jul 31, 2014 8:02 pm 
Offline
ultra-veteran
ultra-veteran
User avatar

Joined: Tue Jan 04, 2005 9:47 pm
Posts: 585
Has thanked: 43 times
Have thanks: 112 times
[out]


Last edited by Wobble on Sun Mar 12, 2017 12:04 pm, edited 1 time in total.

Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 19 posts ]  Go to page 1, 2  Next

All times are UTC + 1 hour


Who is online

Users browsing this forum: No registered users and 4 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group