XeNTaX Forum Index
Forum MultiEx Commander Tools Tools Home
It is currently Mon Nov 20, 2017 3:40 am

All times are UTC + 1 hour


Forum rules


Please click here to view the forum rules



Post new topic Reply to topic  [ 79 posts ]  Go to page 1, 2, 3, 4, 5, 6  Next
Author Message
 Post subject: WWise RIFF Vorbis format?
PostPosted: Wed May 13, 2009 3:28 pm 
Offline
n00b

Joined: Tue May 12, 2009 11:28 pm
Posts: 18
Has thanked: 2 times
Have thanks: 1 time

Hi,

(Recently?) CCP changed all their music into a custom ".ogg" format which turns out to be Vorbis inside a custom RIFF container identified as "WAVE" and has nothing to do with Ogg.

A sample file is here: http://dl.getdropbox.com/u/773354/1.riffvorb

Here's what I know so far:

There are 3 blocks in the RIFF file, "fmt ", "vorb" and "data". "fmt " seems to be the same as in a WAVE file and has valid data but the format is set to 0xFFFF.

Update: "fmt " turned out to be quite a bit larger, there's some extra data there.

"vorb" is 44 bytes, not sure what it is yet. Probably needed to play the file.

"data" seems to be a (raw?) Vorbis stream, you can find the three first Vorbis headers by looking for:

0x01"vorbis" (the identification header)
0x03"vorbis" (the comments header)
0x05"vorbis" (the "codebooks" header)

8 bytes before each header there's a 32-bit int that says how large the header is. The problem I have is that I don't know where each Vorbis audio packet begins and ends.

I've written a small program in C (with very messy code) that extracts the 3 first Vorbis headers and puts them in a proper Ogg file. It's identified as a valid music file but with 0 length as I don't know how to put the Vorbis packets in Ogg pages. You can download it here: http://dl.getdropbox.com/u/773354/extract.zip

Updated:

The format turns out to be "audiokinetics wwise", they use their own container for Vorbis etc. I downloaded their SDK but it doesn't seem possible to play the files without the original project. But at least it helps quite a bit...

You can make the ads go away by registering



Last edited by Anders Bergh on Sat May 23, 2009 7:41 am, edited 3 times in total.

Top
 Profile  
 
 Post subject: Re: EVE Online Vorbis in RIFF container
PostPosted: Wed May 13, 2009 3:51 pm 
Offline
n00b

Joined: Tue May 12, 2009 11:28 pm
Posts: 18
Has thanked: 2 times
Have thanks: 1 time
I uploaded the program I wrote to extract the ogg but it's very ugly code. It was written on Linux and it's used like: make && ./ro music.riffvorbis and creates "anders.ogg". Needs libogg installed.

http://dl.getdropbox.com/u/773354/extract.zip


Top
 Profile  
 
 Post subject: Re: EVE Online Vorbis in RIFF container
PostPosted: Wed May 13, 2009 9:32 pm 
Offline
n00b

Joined: Tue May 12, 2009 11:28 pm
Posts: 18
Has thanked: 2 times
Have thanks: 1 time
(the download links are fixed now)


Last edited by Anders Bergh on Wed May 13, 2009 10:11 pm, edited 1 time in total.

Top
 Profile  
 
 Post subject: Re: EVE Online Vorbis in RIFF container
PostPosted: Wed May 13, 2009 10:01 pm 
Offline
veteran
User avatar

Joined: Sun Jun 08, 2008 10:32 pm
Posts: 127
Location: Developers Country
Has thanked: 1 time
Have thanks: 29 times
Use MEGAUPLOAD.COM, NETLOAD.IN, 4SHARED.COM.
They don't make any problems and are very frendly for nonpaying downloading users.


Top
 Profile  
 
 Post subject: Re: EVE Online Vorbis in RIFF container
PostPosted: Wed May 13, 2009 10:10 pm 
Offline
n00b

Joined: Tue May 12, 2009 11:28 pm
Posts: 18
Has thanked: 2 times
Have thanks: 1 time
asmxtx wrote:
Use MEGAUPLOAD.COM, NETLOAD.IN, 4SHARED.COM.
They don't make any problems and are very frendly for nonpaying downloading users.

http://dl.getdropbox.com/u/773354/extract.zip for my C source code (very ugly)

http://dl.getdropbox.com/u/773354/1.riffvorb for the example file

These work now.


Top
 Profile  
 
 Post subject: Re: WWise RIFF Vorbis format?
PostPosted: Sun Jun 14, 2009 10:06 pm 
Offline
VIP member
VIP member
User avatar

Joined: Sun Apr 17, 2005 11:00 am
Posts: 561
Has thanked: 16 times
Have thanks: 15 times
Thanks for make this unpacker/conversor i tried under Slamd64 and GCC 4.2.3, when i compile i get:
Code:
vorbis.c:110: warning: format '%d' expects type 'int', but argument 2 has type 'size_t'
vorbis.c:170: warning: format '%d' expects type 'int', but argument 2 has type 'ogg_int64_t'

and then if i analyze the ogg
Code:
New logical stream (#1, serial: deadbeef): type vorbis
Vorbis headers parsed for stream 1, information follows...
Version: 0                                               
Vendor: AO; aoTuV b4b [20051117] (based on Xiph.Org's libVorbis)
Channels: 2                                                     
Rate: 48000                                                     

Nominal bitrate: 48,000000 kb/s
Upper bitrate not set         
Lower bitrate not set         
Negative or zero granulepos (-1) on vorbis stream outside of headers. This file was created by a buggy encoder             
...

and
Code:
Total data length: 103036 bytes
        Playback length: 0m:00.000s
        Average bitrate: inf kb/s

_________________
Image


Top
 Profile  
 
 Post subject: Re: WWise RIFF Vorbis format?
PostPosted: Sun Jun 14, 2009 11:48 pm 
Offline
n00b

Joined: Tue May 12, 2009 11:28 pm
Posts: 18
Has thanked: 2 times
Have thanks: 1 time
Savage wrote:
Thanks for make this unpacker/conversor i tried under Slamd64 and GCC 4.2.3, when i compile i get:
Code:
vorbis.c:110: warning: format '%d' expects type 'int', but argument 2 has type 'size_t'
vorbis.c:170: warning: format '%d' expects type 'int', but argument 2 has type 'ogg_int64_t'

and then if i analyze the ogg
Code:
New logical stream (#1, serial: deadbeef): type vorbis
Vorbis headers parsed for stream 1, information follows...
Version: 0                                               
Vendor: AO; aoTuV b4b [20051117] (based on Xiph.Org's libVorbis)
Channels: 2                                                     
Rate: 48000                                                     

Nominal bitrate: 48,000000 kb/s
Upper bitrate not set         
Lower bitrate not set         
Negative or zero granulepos (-1) on vorbis stream outside of headers. This file was created by a buggy encoder             
...

and
Code:
Total data length: 103036 bytes
        Playback length: 0m:00.000s
        Average bitrate: inf kb/s

Yeah, it's broken. I don't know the audio packet boundaries so all the generated .ogg has is the 3 first headers.


Top
 Profile  
 
 Post subject: Re: WWise RIFF Vorbis format?
PostPosted: Mon Jun 15, 2009 9:16 pm 
Offline
VIP member
VIP member
User avatar

Joined: Sun Apr 17, 2005 11:00 am
Posts: 561
Has thanked: 16 times
Have thanks: 15 times
Maybe here we get some clues
http://www.hydrogenaudio.org/forums/ind ... opic=69941 8D

PS: lol, 4 years here :D

_________________
Image


Top
 Profile  
 
 Post subject: Re: WWise RIFF Vorbis format?
PostPosted: Tue Jun 16, 2009 1:08 am 
Offline
n00b

Joined: Tue May 12, 2009 11:28 pm
Posts: 18
Has thanked: 2 times
Have thanks: 1 time
Savage wrote:
Maybe here we get some clues
http://www.hydrogenaudio.org/forums/ind ... opic=69941 8D

PS: lol, 4 years here :D

I will look into all this again and post everything I know so far. Haven't really touched this in a while now...


Top
 Profile  
 
 Post subject: Re: WWise RIFF Vorbis format?
PostPosted: Fri Jun 26, 2009 10:37 pm 
Offline
Site Admin
User avatar

Joined: Wed Jan 15, 2003 6:45 pm
Posts: 4020
Location: Dungeons of Doom
Has thanked: 414 times
Have thanks: 554 times
Savage, thank you for sticking with all of us for 4 years ;)


Top
 Profile  
 
 Post subject: Re: WWise RIFF Vorbis format?
PostPosted: Wed Jul 08, 2009 5:25 am 
Offline
n00b

Joined: Tue May 12, 2009 11:28 pm
Posts: 18
Has thanked: 2 times
Have thanks: 1 time
I have "figured" the format out enough to convert the files to playable Ogg Vorbis. You can find the converter here:

http://andersman.org/~anders/wwiseconv.py

It requires Python 2.5+ and mutagen.

This should work for other games using WWise as well, but I don't have any. Also, when I tried the latest WWise SDK and encoded some Vorbis stuff it seems they changed the format...


Top
 Profile  
 
 Post subject: Re: WWise RIFF Vorbis format?
PostPosted: Sat Sep 12, 2009 9:58 pm 
Offline
n00b

Joined: Tue May 12, 2009 11:28 pm
Posts: 18
Has thanked: 2 times
Have thanks: 1 time
Code:
#!/usr/bin/env python2.6
import os
import sys
import struct
from uuid import UUID

input_file = '/Applications/EVE Online.app/Contents/Resources/transgaming/c_drive/Program Files/CCP/EVE/res/Audio/128526931.ogg'

def parse(stream):
    save = {}

    magic, file_size, file_type = struct.unpack('<4sL4s', stream.read(12))
    print "magic: %s file_size: %d file_type: %s" % (magic, file_size, file_type)

    while stream.tell() <= file_size:
        chunk, size = struct.unpack('<4sL', stream.read(8))
        print "  chunk: '%s' size: %d" % (chunk, size)

        if chunk == 'fmt ':
            (format, channels, samples_per_sec, avg_bytes_per_sec, blockalign,
            bits_per_sample, size, valid_bits, channel_mask, guid) = struct.unpack('<HHLLHHHHL16s', stream.read(size))

            uuid = UUID(bytes=guid)

            print "    format: %02x; channels: %d; rate: %d; avgbytes: %d" % (format, channels, samples_per_sec, avg_bytes_per_sec)
            print "    bits_per_sample: %d size: %d valid_bits: %d channel_mask: %d" % (bits_per_sample, size, valid_bits, channel_mask)
            print "    guid: {%s}" % str(uuid)

            save['rate'] = samples_per_sec

        elif chunk == 'vorb':
            data = stream.read(size)
            samples, a, b, c, d, audio_size, f, g, h, i, j = struct.unpack('<LLLLLLLLLLL', data)
            save['audio_size'] = audio_size
            song_len = samples // save['rate']
            print "    samples: %d audio_size: %d" % (samples, audio_size)
            print "    samples / %d = %0.2f secs" % (save['rate'], song_len)

        elif chunk == 'data':
            orig_pos = stream.tell()

            ident_size, unknown1 = struct.unpack('<LL', stream.read(8))
            ident = stream.read(ident_size)

            comments_size, unknown2 = struct.unpack('<LL', stream.read(8))
            comments = stream.read(comments_size)

            codebooks_size, unknown3 = struct.unpack('<LL', stream.read(8))
            codebooks = stream.read(codebooks_size)

            print "    chunk_size - audio_size = %d" % (size - save['audio_size'])

            print "    ident_size: %d comments_size: %d codebooks_size: %d" % (ident_size, comments_size, codebooks_size)
            print "    unknown1: %d unknown2: %d unknown3: %d" % (unknown1, unknown2, unknown3)

            packets = 0

            while stream.tell() < size + orig_pos:
                packet_size, granule = struct.unpack('<LL', stream.read(8))

                # need to figure out these
                if granule == 0:
                    print "        [%d] granule = %d; size = %d" % (packets, granule, packet_size)
                    print "            %s" % repr(stream.read(packet_size))
                else:
                    stream.seek(packet_size, os.SEEK_CUR)
                packets = packets + 1

            print "    found %d vorbis packets" % packets

            extra = stream.read(size - save['audio_size'])
            print len(extra)

        else:
            print "    skipping chunk"
            stream.seek(size, os.SEEK_CUR)

        print ""

if len(sys.argv) == 2:
    input_file = sys.argv[1]

print "== %s ===========" % input_file
f = open(input_file, 'rb')
parse(f)


Run this on an EVE .ogg file. I'm trying to figure out the format. You can find my initial documentation at: http://bitbucket.org/anders/wwiseconv/wiki/WWise_format

I need to figure out the fields in "vorb" and the 2 initial packets in the "data" chunk (after the 3 vorbis headers).


Top
 Profile  
 
 Post subject: Re: WWise RIFF Vorbis format?
PostPosted: Fri Dec 04, 2009 10:01 am 
Offline
mega-veteran
mega-veteran

Joined: Sun Oct 18, 2009 9:41 pm
Posts: 263
Location: Portland, OR
Has thanked: 3 times
Have thanks: 70 times
Apparently Assassin's Creed 2 uses a later version of this format. I made a converter that converts that to normal Ogg Vorbis. The biggest oddity is that codebooks are stored external to the file. Other than that, maybe you're dealing with the same kind of corner-cutting (leaving out reserved or unneeded fields) that I did, it might be helpful for reference. I doubt it's 100% correct yet, but it pretty much works.
http://hcs64.com/files/ww2ogg00.zip


Top
 Profile  
 
 Post subject: Re: WWise RIFF Vorbis format?
PostPosted: Fri Dec 04, 2009 10:18 am 
Offline
n00b

Joined: Tue May 12, 2009 11:28 pm
Posts: 18
Has thanked: 2 times
Have thanks: 1 time
hcs wrote:
Apparently Assassin's Creed 2 uses a later version of this format. I made a converter that converts that to normal Ogg Vorbis. The biggest oddity is that codebooks are stored external to the file. Other than that, maybe you're dealing with the same kind of corner-cutting (leaving out reserved or unneeded fields) that I did, it might be helpful for reference. I doubt it's 100% correct yet, but it pretty much works.
http://hcs64.com/files/ww2ogg00.zip

I'll have a look, but I downloaded the Wwise SDK and the codebooks are stored within the .ogg files. There is only one difference: in the Vorbis specification there is a set of "time books" (?) and these are simply not included in the Wwise Vorbis files, so you have to parse the whole codebooks then rewrite them but with the times also. That's only 6 bits or so.

Codebooks are different for every encoder and also encoded file, usually, so that you could just embed some other codebooks is quite interesting...

Reading the code at the moment and just saw that you figured out the vorb header!!! Great! I then can fix my new converter (at http://bitbucket.org/anders/wwiseconv) which currently only works for some files, because I reverse engineered the Wwise SDK and put a breakpoint on vorbis_info_init which gets blocksize0 and blocksize1 passed to it!

Stay tuned ;)

PS: I see you parsed the codebooks yourself, perhaps you've noticed I was a bit lazy: I just copied libvorbis and added a "dst" oggpack_buffer to every read function.. so they copy everything, and when they get to the time stuff it just writes that.


Top
 Profile  
 
 Post subject: Re: WWise RIFF Vorbis format?
PostPosted: Fri Dec 04, 2009 11:41 am 
Offline
n00b

Joined: Tue May 12, 2009 11:28 pm
Posts: 18
Has thanked: 2 times
Have thanks: 1 time
I just committed some new changes to my repository, still doesn't work for some files (blocksize_0_pow / blocksize_1_pow seems to be 8 / 11 for these files).

I know the code is really ugly, sorry for that :\



Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 79 posts ]  Go to page 1, 2, 3, 4, 5, 6  Next

All times are UTC + 1 hour


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group