XeNTaX Forum Index
Forum MultiEx Commander Tools Tools Home
It is currently Sat Sep 23, 2017 8:53 am

All times are UTC + 1 hour


Forum rules


Please click here to view the forum rules



Post new topic Reply to topic  [ 84 posts ]  Go to page Previous  1, 2, 3, 4, 5, 6  Next
Author Message
 Post subject: Re: EA Layer 3 Extractor/Decoder
PostPosted: Fri Aug 19, 2016 1:07 am 
Offline
veteran

Joined: Sat Nov 15, 2008 1:31 am
Posts: 83
Has thanked: 0 time
Have thanks: 13 times

There is at least one unknown flag in the EALayer3 stuff (at least in the latest version posted by Zench) and I am getting close to identifying that flag based on the BF3 information.

Maybe I dont find anything new but hey, if I do, great, I will share it with everyone.

You can make the ads go away by registering



Top
 Profile  
 
 Post subject: Re: EA Layer 3 Extractor/Decoder
PostPosted: Fri Aug 19, 2016 4:42 pm 
Offline
double-veteran
double-veteran

Joined: Tue Mar 24, 2015 8:12 pm
Posts: 1178
Has thanked: 19 times
Have thanks: 548 times
Since all sounds from BF3 can be decoded properly without that flag, then I'd say this flag is not needed.


Top
 Profile  
 
 Post subject: Re: EA Layer 3 Extractor/Decoder
PostPosted: Sat Aug 20, 2016 12:18 am 
Offline
veteran

Joined: Sat Nov 15, 2008 1:31 am
Posts: 83
Has thanked: 0 time
Have thanks: 13 times
There are many EA games using the exact same EALayer3 codec (the one labeled 6/L32P/7/L32S), any one of those could have EALayer3 audio with that particular bit set.

It just happens that the Battlefield 3 binary is the first one to show up with enough debugging information to make it easier to reverse engineer the code.
The point here is not what some specific subset of EALayer3 encoded audio looks like but what the decoder does for all possible inputs so that it can decode every possible file encoded with the 3 codecs in question (the one labeled 5/EL31, the one labeled 6/L32P and the one labeled 7/L32S) in exactly the same way the official decoder decodes them.

I have already identified at least one case where there is data being read by the official code but not by the ealayer3 code although I haven't quite figured out exactly which combination of header bits in the "EALayer3 Header" (the one with the version/sample rate/mode in it) causes this data to be read yet.

In any case fiddling with this stuff is FUN :)


Top
 Profile  
 
 Post subject: Re: EA Layer 3 Extractor/Decoder
PostPosted: Sat Aug 20, 2016 6:47 am 
Offline
double-veteran
double-veteran

Joined: Tue Mar 24, 2015 8:12 pm
Posts: 1178
Has thanked: 19 times
Have thanks: 548 times
jfwfreo wrote:
I have already identified at least one case where there is data being read by the official code but not by the ealayer3


Can you send me that file? You know, I just can't believe that can be. I converted dozens of games, and it never happened.


Top
 Profile  
 
 Post subject: Re: EA Layer 3 Extractor/Decoder
PostPosted: Sat Aug 20, 2016 11:25 am 
Offline
veteran

Joined: Sat Nov 15, 2008 1:31 am
Posts: 83
Has thanked: 0 time
Have thanks: 13 times
I haven't seen a file that has this particular information in it. But that doesn't mean it doesn't exist. What matters to me is that every single piece of data that the official decoder will read and decode is also decoded by the ealayer3 code so it can be 100% accurate in case there IS audio out there that uses this stuff.

I doubt you have seen every EALayer3 encoded file that has ever been produced...

And again, like I said, its fun so even if no files exist using this particular data, figuring out all this stuff is still interesting to me :)

Question: Are you aware of any EALayer3 encoded files that have a value in the MPEG version field other than 3? (representing MPEG version 1) If you know of such files, please tell me where to find them (what game, whatever) as there are some interesting things in the code that may be triggered only for files where the version field != 3 and if I get the files I can see what they look like and maybe answer some questions.


Top
 Profile  
 
 Post subject: Re: EA Layer 3 Extractor/Decoder
PostPosted: Sat Aug 20, 2016 1:50 pm 
Offline
double-veteran
double-veteran

Joined: Tue Mar 24, 2015 8:12 pm
Posts: 1178
Has thanked: 19 times
Have thanks: 548 times
jfwfreo wrote:
I doubt you have seen every EALayer3 encoded file that has ever been produced...


Yes, but statistically, after I've seen 1000s of files, chances I've seen all types are closing to 99%.

Ok, you can find MPEG2 in low quality SFXs like explosions or gunshots, usually in sound banks, for example in Dead Space 2. But they were all converted properly. Believe me, I listened to them all. I can suppose that official decoder may apply some filter for such low-freq files, but this is not about the actual decoding process or flags.

Also, nowadays devs are more and more returning to ADPCM and even using PCM or FLAC, so I don't think low quality mpeg such as 22000 hz is any useful now.


Top
 Profile  
 
 Post subject: Re: EA Layer 3 Extractor/Decoder
PostPosted: Sat Aug 20, 2016 2:02 pm 
Offline
veteran

Joined: Sat Nov 15, 2008 1:31 am
Posts: 83
Has thanked: 0 time
Have thanks: 13 times
Can you point me to an example of an EALayer3 compressed file with a version field not equal to 3? Yes I know you say they all convert properly but I still want to look at it anyway to see what it looks like inside.
I am also after an EALayer3 compressed file of type EL31 (also known as "type 5") that has uncompressed data following the compressed data (should have 0xEE in the first byte of the granule rather than 0x00) as its the other thing I dont have an example of at this point.

Oh and you say you have seen/converted many many different EALayer3 compressed files. Have you seen/converted the EALayer3 audio connected to the PathMusic data in Red Alert 3, Red Alert 3 Uprising and Command & Conquer 4 Tiberian Twilight? Specifically the large blob that contains multiple different pieces of EALayer3 compressed audio? (and if you HAVE been able to convert the audio data inside it, please tell me how you did it as I would love to know :)


Top
 Profile  
 
 Post subject: Re: EA Layer 3 Extractor/Decoder
PostPosted: Sun Aug 21, 2016 4:39 am 
Offline
veteran

Joined: Sat Nov 15, 2008 1:31 am
Posts: 83
Has thanked: 0 time
Have thanks: 13 times
Does EALayer3 (either the original code or your changed code) handle EALayer3 audio using MPEG-2 Low Sampling Frequency? Do any EALayer3 files exist that use this?

BINGO, I figured it out. Turns out the "unknown data" I was staring at is the MPEG Side Info and I just found the code that reads it in the EALayer3 codebase so no its not "data ealayer3 doesn't read" like I first thought. It was confusing me until I realized the reason the ealayer3 code wasn't actually making use of the "side info" is that it doesn't actually decode MPEG audio, it just passes it to mpg123 (which DOES make use of the side info).

So yeah I dont need an example of MPEG-2 LSF anymore now that I understand what is really going on in the decoder (or an example of EALayer3 audio that has version != 3).
I dont need a "type 5" file with "uncompressed samples" either.

But if you know how to decode the "path music" large blobs from RA3/Uprising/C&C4, I would love to know how...


Top
 Profile  
 
 Post subject: Re: EA Layer 3 Extractor/Decoder
PostPosted: Sun Aug 21, 2016 7:06 am 
Offline
double-veteran
double-veteran

Joined: Tue Mar 24, 2015 8:12 pm
Posts: 1178
Has thanked: 19 times
Have thanks: 548 times
Good that you got it. You see, in "mp3" mode ealayer3 doesn't even decode anything, it just puts the parts of data to standard mp3.

jfwfreo wrote:
But if you know how to decode the "path music" large blobs from RA3/Uprising/C&C4, I would love to know how...


Can you send me example of such file, because I don't quite understand what are you talking about.


Top
 Profile  
 
 Post subject: Re: EA Layer 3 Extractor/Decoder
PostPosted: Sun Aug 21, 2016 7:57 am 
Offline
veteran

Joined: Sat Nov 15, 2008 1:31 am
Posts: 83
Has thanked: 0 time
Have thanks: 13 times
Its a large data file containing a bunch of separate pieces of EALayer3 audio.
But if you dont know what PathMusic or PathMusicTrack or PathMusicMap or .mpf or .mus files are, you probably dont know how to unpack this data.

https://mega.nz/#!u50nzRQK!ZWBhTO8awlhe ... ALxFDxgQL8 is an example of one of these files with all the audio chunks in it in case you want to have a look :)


Top
 Profile  
 
 Post subject: Re: EA Layer 3 Extractor/Decoder
PostPosted: Sun Aug 21, 2016 8:32 am 
Offline
double-veteran
double-veteran

Joined: Tue Mar 24, 2015 8:12 pm
Posts: 1178
Has thanked: 19 times
Have thanks: 548 times
jfwfreo wrote:
But if you dont know what PathMusic or PathMusicTrack or PathMusicMap or .mpf or .mus files are, you probably dont know how to unpack this data.


Don't underestimate me ;) If you check my profile, you can see I'm a reverse engineering specialist.

That was very easy. This file is a sound bank containing 17 sounds. In the beginning you see a table with all of them. It contains size, data offset, hash, header size and header offset for all sounds. Then go all sounds according to that table. You can take all of this info, get the data, and decode individual files with ealayer3.


Top
 Profile  
 
 Post subject: Re: EA Layer 3 Extractor/Decoder
PostPosted: Sun Aug 21, 2016 8:56 am 
Offline
veteran

Joined: Sat Nov 15, 2008 1:31 am
Posts: 83
Has thanked: 0 time
Have thanks: 13 times
Can you show me the data structure(s) needed for this file? Its not obvious from your description (and looking at the file in a hex editor) just what size the fields are.
Is there a count stored anywhere of how many tracks are in the .mus file?


Last edited by jfwfreo on Sun Aug 21, 2016 9:04 am, edited 1 time in total.

Top
 Profile  
 
 Post subject: Re: EA Layer 3 Extractor/Decoder
PostPosted: Sun Aug 21, 2016 9:00 am 
Offline
double-veteran
double-veteran

Joined: Tue Mar 24, 2015 8:12 pm
Posts: 1178
Has thanked: 19 times
Have thanks: 548 times
all fields are 32-bit words.

Count is at offset 4. Table starts from 0x1C

Table fields:

hash
number of file
header offset (multiply by 0x10)
data offset (multiply by 0x80)
header size
data size
zero (or unknown)

You don't actually need headers, because sample freq and channel number already contained in mp3 data


Top
 Profile  
 
 Post subject: Re: EA Layer 3 Extractor/Decoder
PostPosted: Sun Aug 21, 2016 9:16 am 
Offline
veteran

Joined: Sat Nov 15, 2008 1:31 am
Posts: 83
Has thanked: 0 time
Have thanks: 13 times
Are you sure you dont mean 0x28? Looks that way to me with the first hash being C5C98A9C
And do you have any idea what the other fields in the file header (other than the count) might be?

Oh and I see what you mean about EALayer3 being very close to MP3, I see large chunks of the low-level decoding stuff (DecodeHuffman, Dequantize, others) shared between the EALayer3 decoder and the MP3 decoder. I do wonder why they invented a format that is almost but not quite MP3...


Top
 Profile  
 
 Post subject: Re: EA Layer 3 Extractor/Decoder
PostPosted: Sun Aug 21, 2016 9:57 am 
Offline
double-veteran
double-veteran

Joined: Tue Mar 24, 2015 8:12 pm
Posts: 1178
Has thanked: 19 times
Have thanks: 548 times
jfwfreo wrote:
Are you sure you dont mean 0x28?


Oh, sure its 0x28. A little misplace when writing. The other fields must be something to connect this data to other game files. And I don't have them.

Well, their format IS actually normal mp3. The only thing they did is storing mp3 data in their own structure. Why? No idea.



Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 84 posts ]  Go to page Previous  1, 2, 3, 4, 5, 6  Next

All times are UTC + 1 hour


Who is online

Users browsing this forum: No registered users and 2 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group