XeNTaX Forum Index
Forum MultiEx Commander Tools Tools Home
It is currently Fri May 25, 2018 1:48 am

All times are UTC + 1 hour


Forum rules


Please click here to view the forum rules



Post new topic Reply to topic  [ 14 posts ] 
Author Message
 Post subject: Maplestory 2
PostPosted: Sun May 13, 2018 5:18 am 
Offline
beginner
User avatar

Joined: Thu Dec 23, 2010 9:49 am
Posts: 26
Has thanked: 6 times
Have thanks: 15 times














You can make the ads go away by registering

Been digging into the closed beta lately and I have yet to see one central cohesive place where anyone is documenting any of their efforts so in hopes of starting some cohesive efforts before I have to divide myself away for finals. Hoping someone can pick things up with this info. Open to more questions as well.

At the moment the best IDA pro databases you can make for yourself are by suspending Maplestory2.exe and opening the process in IDA Pro to do a live analysis and eventually saving your database once you've analyzed enough of the relevant modules(such as kernel32 and mfc90) or else you'll run into calls into import pools that lead to seemingly random addresses(be sure to analyse the mfc90 and kernel32 modules and so forth before saving your database as there are no IATs).

Here are some links. Note that these are transfer.sh links that will expire in about 2 weeks.

https://transfer.sh/157KM2/Library.m2h
https://transfer.sh/f3m0i/Library.m2d
https://transfer.sh/KRGUE/Movie.m2h
https://transfer.sh/bXdPg/Movie.m2d
https://transfer.sh/fEmN3/Shaders.m2h
https://transfer.sh/Aaqw2/Shaders.m2d
https://transfer.sh/Tw455/asset-web-metadata.m2h
https://transfer.sh/jR3Ak/asset-web-metadata.m2d

Archives come in couples of "m2h" and "m2d" files such that there is a "Header" and the Data that the header is binded to. Format is different depending on the 4-byte header as well as the size of the headers which I have yet to map out fully due to vtable hell but the sizes are all determinant.
    Magic - Size
    ---------------
    MS2F - 0x3C
    NS2F - 0x34
    OS2F - 0x38
    PS2F - 0x38
After these 4 bytes is the header bytes. And then followed by a base64 stream:
Image

I've traced enough to get several of the XOR keys and the pools of 128 AES Keys and Initialization Vectors which it seems to select at run time.
Image

The game uses CryptoPP as its encryption middleware and exposes plenty of strings and RTTI symbols to find out where things are at. The above disassembly is CryptoPP's usual Source-Sink pattern of decrypting a stream concisely:
Image
Image

In the above pic you'll see it setting up a string source, base64 decoder, some decryptor, and a string sink and a call to what turns out to be SetKeyWithIV where it picks from a "vault" of 128 possible(the `& 0x7F` is the same as a `% 128`) 32-byte(256 bit) keys and a 16-byte(128 bit) initialization vector. The way it selects one of these 128 keys is based on one of the fields of the header(Internally named 'PackFileHeaderVer1' for the 'MS2F' files) using the 64-bit integer at 0x24. After the initial base64-decoding. The stream is optionally decrypted with a xor pad and decompressed using zlib's "inflate".

Here's all the keys.

https://transfer.sh/CFU1d/MS2FIVVault.bin
https://transfer.sh/njxur/MS2FKeyVault.bin

https://transfer.sh/gRr2l/NS2FXORKey.bin
https://transfer.sh/q6k2p/OS2FXORKey.bin
https://transfer.sh/NhpYd/PS2FXORKey.bin
https://transfer.sh/PqM52/MS2FXORKey.bin

_________________
Aka Wunkolo


Top
 Profile  
 
 Post subject: Re: Maplestory 2
PostPosted: Sun May 13, 2018 11:49 am 
Offline
M-M-M-Monster veteran
M-M-M-Monster veteran

Joined: Wed Mar 31, 2010 6:54 am
Posts: 1622
Has thanked: 61 times
Have thanks: 778 times
Do you have EN CBT client?


Top
 Profile  
 
 Post subject: Re: Maplestory 2
PostPosted: Mon May 14, 2018 8:11 am 
Offline
beginner
User avatar

Joined: Thu Dec 23, 2010 9:49 am
Posts: 26
Has thanked: 6 times
Have thanks: 15 times
Ekey wrote:
Do you have EN CBT client?

Here's all the client data for the english closed beta.
https://mega.nz/#!eK4T3LbL!ejsWFNipuow1 ... P0mbXxMJyI

_________________
Aka Wunkolo


Top
 Profile  
 
 Post subject: Re: Maplestory 2
PostPosted: Mon May 14, 2018 12:43 pm 
Offline
M-M-M-Monster veteran
M-M-M-Monster veteran

Joined: Wed Mar 31, 2010 6:54 am
Posts: 1622
Has thanked: 61 times
Have thanks: 778 times
Thanks for client. As I see for NS2F, OS2F and PS2F keys and vectors are generated individually.

Code:
int __stdcall sub_4A04B0(int a1, int a2, int a3, int a4, int a5)
{
  switch ( a1 )
  {
    case 0x4632534D:
      return sub_49FBF0(a2, a3, a4, a5);
    case 0x4632534E:
      return sub_49FE20(a2, a3, a4, a5);
    case 0x4632534F:
      return sub_4A0050(a2, a3, a4, a5);
    case 0x46325350:
      return sub_4A0280(a2, a3, a4, a5);
  }
  return 0;
}


MS2F (Static keys)

Image

NS2F (Dynamic keys)

Image

OS2F (Dynamic keys)

Image

PS2F (Dynamic keys)

Image

That keys generates this call int sub_48EED0() or it's more like pointers

Image

Seems j_mfc90_265 is a malloc.

Code:
buffer = (char*) malloc (512);


Top
 Profile  
 
 Post subject: Re: Maplestory 2
PostPosted: Mon May 14, 2018 8:24 pm 
Offline
beginner
User avatar

Joined: Thu Dec 23, 2010 9:49 am
Posts: 26
Has thanked: 6 times
Have thanks: 15 times
I noticed these too. It generates the other keys based on some xoring of the static MS2F keys and the associated static XorKey for the NS2F and so forth though I did not find any files that used these headers. They are allocated on the heap and will have to be dumped at runtime if we want them.

I made a megafolder with the keys and client.

https://mega.nz/#F!DbxEAL5S!MvLhvAcyusMF4kp6_qBXYA

I was in the middle of mapping out the vtables and logic for MS2F files(Internally called PackFileHeaderVer1) before I had to part. This is what I got in my little utility library before I had to part with my main workstation.

Code:
MapleStory2.exe:015CC9D8 dd offset ??_R4?$PackFileStream@VCPackStreamVer1@@@@6B@ ; const PackFileStream<CPackStreamVer1>::`RTTI Complete Object Locator'
MapleStory2.exe:015CC9DC ; const PackFileStream<class CPackStreamVer1>::`vftable'
MapleStory2.exe:015CC9DC ??_7?$PackFileStream@VCPackStreamVer1@@@@6B@ dd offset sub_579430
MapleStory2.exe:015CC9DC                                         ; DATA XREF: CPackStreamVer1_Ctor+27↑o
MapleStory2.exe:015CC9E0 dd offset ??_R4?$PackFileStream@VCPackStreamVer1@@@@6B@_0 ; const PackFileStream<CPackStreamVer1>::`RTTI Complete Object Locator'
MapleStory2.exe:015CC9E4 ; CPackStreamVer1Vtable PackFileStream<CPackStreamVer1>::`vftable'
MapleStory2.exe:015CC9E4 ??_7?$PackFileStream@VCPackStreamVer1@@@@6B@_0 CPackStreamVer1Vtable <offset CPackStreamVer1Vtable__Dtor, \
MapleStory2.exe:015CC9E4                                         ; DATA XREF: CPackStreamVer1_Ctor+21↑o
MapleStory2.exe:015CC9E4                        offset CPackStreamVer1Vtable__GetHeaderSize, \
MapleStory2.exe:015CC9E4                        offset CPackStreamVer1Vtable__GetBufferPtr, \
MapleStory2.exe:015CC9E4                        offset CPackStreamVer1Vtable__field_C, \
MapleStory2.exe:015CC9E4                        offset CPackStreamVer1Vtable__GetCompressedSize?,\
MapleStory2.exe:015CC9E4                        offset CPackStreamVer1Vtable__field_14, \
MapleStory2.exe:015CC9E4                        offset CPackStreamVer1Vtable__GetLength?, \
MapleStory2.exe:015CC9E4                        offset CPackStreamVer1Vtable__GetVersion?, \
MapleStory2.exe:015CC9E4                        offset CPackStreamVer1Vtable__field_20, \
MapleStory2.exe:015CC9E4                        offset CPackStreamVer1Vtable__field_24, \
MapleStory2.exe:015CC9E4                        offset CPackStreamVer1Vtable__SetMagic, \
MapleStory2.exe:015CC9E4                        offset CPackStreamVer1Vtable__field_2C, \
MapleStory2.exe:015CC9E4                        offset CPackStreamVer1Vtable__field_30, \
MapleStory2.exe:015CC9E4                        offset CPackStreamVer1Vtable__field_34, \
MapleStory2.exe:015CC9E4                        offset CPackStreamVer1Vtable__field_38, \
MapleStory2.exe:015CC9E4                        offset CPackStreamVer1Vtable__field_3C, \
MapleStory2.exe:015CC9E4                        offset CPackStreamVer1Vtable__field_40, \
MapleStory2.exe:015CC9E4                        offset CPackStreamVer1Vtable__field_44>


The header structures do not consider the 4 Magic bytes.
Code:
union PackFileHeaderVer1
{
public:
        Util::Field<0x00, std::uint32_t> Unknown00;
        Util::Field<0x04, std::uint64_t> Unknown04;
        Util::Field<0x0C, std::uint64_t> Unknown0C; // Compressed Size?
        Util::Field<0x14, std::uint64_t> Unknown14;
        Util::Field<0x1C, std::uint64_t> Unknown1C; // Size?
        Util::Field<0x24, std::uint64_t> Unknown24; // Version?
        Util::Field<0x2C, std::uint32_t> Unknown2C;
        Util::Field<0x30, std::uint32_t> Unknown30;
        Util::Field<0x34, std::uint64_t> Unknown34;

private:
        Util::Padding<0x3C> Pad;
};


The "version" flag(named it that might it's above 128 at times and probably means something other than just a straight up "KeyIndex") might be what is used to determine which of the 128 keys that it uses from the Key/IV LUT(after a % 128) from what I've seen.

Here's the full decryption pseudo code which i made some guesses for.
Code:
CFileSystem = this;
  PackFile = PackFile_1;
  Length = 0;
  this[24] = 0;
  v8 = PackFile->vtable->GetLength_(PackFile);
  mfc_Alloc_0(v8);
  Buffer = v9;
  v32 = v9;
  Size = (PackFile->vtable->GetLength_)(PackFile, &Length, 0);
  ReadFile(FileHandle, Buffer, Size, BytesRead, Overlapped);
  IsCompressedFunc? = PackFile->vtable->GetCompressedSize_;
  PackFile_1 = 0;
  if ( !(*(IsCompressedFunc?(PackFile, &FileHandle) + 3) & 1) )
  {
    KeyIndex_1 = PackFile->vtable->GetVersion_(PackFile);
    v14 = (*(*CFileSystem[55] + 8))(            // AES decrypt
            CFileSystem[57],
            Buffer,
            Length,
            &PackFile_1,
            KeyIndex_1);
    if ( v14 && PackFile_1 )
    {
      mfc90_free(v32);
      v32 = 0;
      goto LABEL_7;
    }
LABEL_32:
    mfc90_free(v32);
    return 0;
  }
  KeyIndex = PackFile->vtable->GetVersion_(PackFile);
  v14 = (*(*CFileSystem[56] + 8))(              // Xor Decrypt
          CFileSystem[57],
          Buffer,
          Length,
          &PackFile_1,
          KeyIndex);
  if ( !v14 || !PackFile_1 )
    goto LABEL_32;
LABEL_7:
  if ( v14 != PackFile->vtable->GetVersion_(PackFile) || v16 )
  {
    if ( v32 )
    {
      mfc90_free(v32);
      return 0;
    }
    return 0;
  }
  if ( *PackFile->vtable->GetCompressedSize_(PackFile, &FileHandle) )// Compression
  {
    v21 = Compressed;
    v22 = 0;
    CFileSystem[26] = *a4;
    CFileSystem[27] = v21;
    if ( v21 )
    {
      while ( 1 )
      {
        if ( !CFileSystem[24] )
        {
          v23 = Length;
          v24 = PackFile_1 + v22;
          v22 += Length;
          v25 = CFileSystem[38] < Length;
          CFileSystem[38] -= Length;
          CFileSystem[23] = v24;
          CFileSystem[24] = v23;
          CFileSystem[39] -= v25;
        }
        if ( *(CFileSystem + 19) )
          v26 = 0;
        else
          v26 = 4;
        v27 = zlib_inflate(CFileSystem + 23, v26);
        if ( v27 == -5 )
        {
          mfc90_free(PackFile_1);
          return 0;
        }
        if ( v27 )
          break;
        if ( !CFileSystem[27] )
        {
          mfc90_free(PackFile_1);
          return 0;
        }
      }
      if ( v27 == 1 )
      {
        *a6 = Compressed - CFileSystem[27];
        goto LABEL_28;
      }
      mfc90_free(PackFile_1);
      result = 0;
    }
    else
    {
      v28 = a6;
      v29 = HIDWORD(Compressed);
      *a6 = Compressed;
      v28[1] = v29;
LABEL_28:
      mfc90_free(PackFile_1);
      result = 1;
    }
  }
  else
  {
    v18 = *(CFileSystem + 19) - Length;
    if ( v18 >= 0 )
    {
      *(CFileSystem + 19) = v18;
    }
    else
    {
      CFileSystem[38] = 0;
      CFileSystem[39] = 0;
    }
    v19 = a6;
    a6[1] = 0;
    v20 = a4;
    *v19 = v14;
    j_msvcr90_memcpy(*v20, PackFile_1, v14);
    mfc90_free(PackFile_1);
    result = 1;
  }
  return result;

_________________
Aka Wunkolo


Top
 Profile  
 
 Post subject: Re: Maplestory 2
PostPosted: Mon May 14, 2018 10:14 pm 
Offline
M-M-M-Monster veteran
M-M-M-Monster veteran

Joined: Wed Mar 31, 2010 6:54 am
Posts: 1622
Has thanked: 61 times
Have thanks: 778 times
DEElekgolo wrote:
I did not find any files that used these headers

Xml.m2h used NS2F. About headers: is not to hard

Code:
struct MSPHeader
{
   uint64_t   dwMagic;
};

struct MS2FSubHeader
{
   uint64_t   dwTOCZSize; //compressed size (zlib) also it's a KeyIndex
   uint64_t   dwTOCESize; //encoded size
   uint64_t   dwFileListSize;  //uncompressed size
   uint64_t   dwFileListZSize;  //compressed size (zlib) also it's a KeyIndex
   uint64_t   dwFileListESize; //encoded size
   uint64_t   dwTotalFiles;
   uint64_t   dwTOCSize; //uncompressed size
};

struct NS2FSubHeader
{
   uint32_t   dwTotalFiles;
   uint64_t   dwTOCZSize; //compressed size (zlib) also it's a KeyIndex
   uint64_t   dwTOCESize; //encoded size
   uint64_t   dwTOCSize; //uncompressed size
   uint64_t   dwFileListZSize; //compressed size (zlib) also it's a KeyIndex
   uint64_t   dwFileListESize; //encoded size
   uint64_t   dwFileListSize; //uncompressed size
};


Example on asset-web-metadata file:

1) Read data by dwFileListESize = 340 bytes
2) Decode and decrypt by Base64 + AES and you got size equal dwFileListZSize = 254 bytes
3) Decompress it (dwFileListSize = 480 bytes) and we got list of filenames like

Code:
1,application.nt
2,awebckpt.nt
3,awebinst.nt
4,canonical.nt
5,cn.nt
6,dds.nt
7,emergent-flat-model.nt
8,emergent-world.nt
9,enums.nt
10,fx-shader-compiled.nt
11,gamebryo-animation.nt
12,gamebryo-scenegraph.nt
13,gamebryo-sequence-file.nt
14,image.nt
15,kr.nt
16,llid.nt
17,logpath.nt
18,lua-behavior.nt
19,model.nt
20,name.nt
21,nuts.nt
22,png.nt
23,precache.nt
24,relpath.nt
25,script.nt
26,shader.nt
27,soup.nt
28,x-shockwave-flash.nt
29,x-world.nt


Repeat for TOC table

1) Read data by dwTOCESize = 756 bytes
2) Decode and decrypt by Base64 + AES and you got size equal dwTOCZSize = 565 bytes
3) Decompress it (dwTOCSize = 1392 bytes) and we got table of contents (see attach)

Code:
struct MSPTocEntry
{
   uint32_t   dwNull;
   uint32_t   dwFileID; //equal to the number from the filelist
   uint64_t   dwUnknown; //dunno what this (flags ?) : always 0xEE000009
   uint64_t   dwOffset;
   uint64_t   dwESize; //encoded size
   uint64_t   dwZSize; //compressed size
   uint64_t   dwSize; //uncompressed size
};


Attachments:


You do not have the required permissions to view the files attached to this post. Register to gain access.



Last edited by Ekey on Mon May 21, 2018 12:00 am, edited 2 times in total.

Top
 Profile  
 
 Post subject: Re: Maplestory 2
PostPosted: Tue May 15, 2018 1:39 am 
Offline
beginner
User avatar

Joined: Thu Dec 23, 2010 9:49 am
Posts: 26
Has thanked: 6 times
Have thanks: 15 times
Looks solid!
Can't wait to dive in once I get back and I can make a little general purpose extractor.
Looks like repackaging is even possible. Back when this was a Korea-only game some others had made an English translation patch I believe that involved edits to the m2h and m2d files.

_________________
Aka Wunkolo


Top
 Profile  
 
 Post subject: Re: Maplestory 2
PostPosted: Wed May 16, 2018 3:27 pm 
Offline
ultra-n00b

Joined: Wed May 16, 2018 2:55 pm
Posts: 2
Has thanked: 1 time
Have thanks: 0 time
Keep us updated on the extractor.

I spent all night trying to figure out how to get into those files, but it's quite clearly way above my head.


Top
 Profile  
 
 Post subject: Re: Maplestory 2
PostPosted: Thu May 17, 2018 8:04 am 
Offline
beginner
User avatar

Joined: Thu Dec 23, 2010 9:49 am
Posts: 26
Has thanked: 6 times
Have thanks: 15 times
Wedged in some time to make a quick little decryptor. Can fully open MS2F m2h/m2d files and read/decrypt file data. Hoping to aim it to fully "mounting" the Data folder and recreating the internal virtual file system that "CFileSystem" implements

Image
Image
Image
Image

ATM I don't have the keys for N/O/PS2F files as they are generated at runtime as a multidimensional array though the generation for the keys can be pretty trivially recreated.

Some xoring and swizzling to the effect of:
Image
Which is a ( (i + 5 ) % 8 ) offset from the XOR key lookup. Would probably be much quicker to dump the multidimensional array from memory( which is a pointer to 512 bytes which is 128 pointers to the keys and IVs and such ).

Since the beta is over it might be harder to do some runtime analysis to dump the keys from memory but running the exe directly unpacks it enough that it can be derived. Something I don't have the time atm for but would allow for `NS2F` files and such to be handled.

_________________
Aka Wunkolo


Top
 Profile  
 
 Post subject: Re: Maplestory 2
PostPosted: Thu May 17, 2018 10:16 am 
Offline
beginner
User avatar

Joined: Thu Dec 23, 2010 9:49 am
Posts: 26
Has thanked: 6 times
Have thanks: 15 times
some dumps from Image.m2h/m2d

Image
Image
Image
Image
Image

_________________
Aka Wunkolo


Top
 Profile  
 
 Post subject: Re: Maplestory 2
PostPosted: Thu May 17, 2018 11:33 am 
Offline
ultra-n00b

Joined: Wed May 16, 2018 2:55 pm
Posts: 2
Has thanked: 1 time
Have thanks: 0 time
Will you be sharing the tool once it's ready?


Top
 Profile  
 
 Post subject: Re: Maplestory 2
PostPosted: Thu May 17, 2018 1:48 pm 
Offline
M-M-M-Monster veteran
M-M-M-Monster veteran

Joined: Wed Mar 31, 2010 6:54 am
Posts: 1622
Has thanked: 61 times
Have thanks: 778 times
Looks good, keep going :)


Top
 Profile  
 
 Post subject: Re: Maplestory 2
PostPosted: Fri May 18, 2018 4:29 am 
Offline
beginner
User avatar

Joined: Thu Dec 23, 2010 9:49 am
Posts: 26
Has thanked: 6 times
Have thanks: 15 times
Ewol wrote:
Will you be sharing the tool once it's ready?

Sure. Though I am developing this remotely in a linux environment(compatible with windows but will involve some hoop-jumping to get it to build. Uses CMake and CryptoPP).
I can post the source code once its ready for the public and I can just post a fully dumped version of the game for all to pick through.

At the moment I just need to find time to get NS2F and O/PS2F keys to get my program to recursively dump and flatten out all possible m2h/m2d files it runs into.

_________________
Aka Wunkolo


Top
 Profile  
 
 Post subject: Re: Maplestory 2
PostPosted: Tue May 22, 2018 10:55 pm 
Offline
ultra-n00b

Joined: Sun Apr 03, 2016 11:39 am
Posts: 7
Has thanked: 3 times
Have thanks: 2 times
I uploaded the tools and library that I created when I made the english patches. There is an extractor and a creator tool on my github: https://github.com/Miyuyami/MS2Tools.
Both of them use the library I created which is here: https://github.com/Miyuyami/MS2Lib. This is where you can also find the NS2F, OS2F and PS2F IV and Key pairs that are missing from this thread.

I hope that those tools are not too scuffed/broken and that will help the people interested in this. :)


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 14 posts ] 

All times are UTC + 1 hour


Who is online

Users browsing this forum: No registered users and 5 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group