READ THE RULES: Click here

Follow us on Facebook: https://www.facebook.com/xentax/ :)

Okami HD PC - Texts files

Read or post about compression. And decompression. Or ask questions how to decompress your files.
innocentius92
n00b
Posts: 14
Joined: Sat Dec 16, 2017 9:10 am
Has thanked: 13 times

Okami HD PC - Texts files

Post by innocentius92 » Sat Dec 16, 2017 10:30 am

hi guys, I'm new here, sorry if this does not go in this section, but someone could help me find the texts and how unpack the files of the game? it's for a language translation. Some help?

ffgriever
beginner
Posts: 28
Joined: Fri Dec 15, 2017 5:21 pm
Has thanked: 1 time
Been thanked: 27 times

Re: Okami HD PC - Texts files

Post by ffgriever » Sat Dec 16, 2017 1:26 pm

The first thing you need to realize is that Okami was initially a Wii (and then PS2) game. This means very strict memory and computing power constraints. That's one of many reasons why every format was developed to need the least possible memory footprint and the least possible processing. An XML parser or some other text parser could add a hundred or even few hundreds kB to the application size, then it would require a lot of memory for intermediate buffers before it's processed to the form that can be displayed directly. With small and simple games that's not an issue, but becomes one once you do something as big as one of the AAA games. That's why most games of that era (and before) opted mostly for relative pointers (which sometimes would be changed on the fly to absolute memory addresses), character substitution (containing only the used character glyphs and assigning them numbers in order of appearance in font table or font image), bitmap fonts, etc. That meant you had to place a file in memory only once, sometimes do some minimal processing (like pointer recalculation) and then use the values as they are, without any intermediates.

"Take a value CHAR from text file, take values from font description file at position CHAR*DESC_SIZE, use the values to determine X, Y, W, H coordinates of the glyphs to display, print a glyph from the font texture with given coordinates". That's very simplistic description of what usually happens in PSX/PS2 games (was a little different in nes/snes era, due to tiles, memory addressing, etc.).

With that mindset it will be much easier to take on the Okami file formats.

Anyway, to the point. ~90% of text in Okami is contained within MSD files, which in turn are contained in binary containers alongside many other files. The binary containers are usually one per location/menu/etc. For example st1/r120.bin is an intro stage. You know, "A long time ago in a galaxy far, far away..." or something along the lines, it's been a while. The file formats are VERY straightforward, probably among the most simple ones I've ever found in a game.

These are very similar to PS2 files, with one exception being that at pointer-0x20 location there is a metric with file type and stage name which wasn't present in PS2 version. Which in turn makes all the actuall file sizes next_pointer-previous_pointer-0x20 (except for the very last one, which just spans to the end). From the game perspective it doesn't matter at all (remember, it just loads it and uses the data, it doesn't dissect it into files).

So the file starts with 32bit number of files within container. Then a 32bit pointers to all the files, followed by 32bit magic strings describing file formats. What's interesting for you is the MSD file (or files).

As above, it starts with number of dialogs within a file (doesn't mean number of windows, but full dialogs, sometimes with multiple characters). Then come pointers. Each pointer is either 64bit or 32bit followed by an unknown/reseved 32bit value (I don't recall ever seeing it used). Then at each pointer relative to file start a dialog starts. Each character (or command value) consists of 16bit (two bytes). You'll need to assign characters to the values, but eg. 0xA301 (or 0x01 0xA3) is an action of waiting for gamepad/keyboard entry to move the dialog forward, 0xA101 is a "clearscreen" command, etc.

If you're about to edit Japanese files, please note, that there are also special commands to enable furigana and the text will be intertwined with furigana text. In Japanese version there will also be a lot more images in the files. That's because main Japanese font contains only numbers, romaji, kanas, few hundred most popular kanjis and some other characters. If a particular stage file uses kanjis/other characters not present in the main font file, the font pages are added to the stage file and accessed through special character range, which essentially means that for different stages a particular 16bit value may actually mean other characters. There are few cases when this is used for English version, too.

Some text is contained within other binary files with other formats, on PS2 a few labels were also hardcoded into executable, but the MSD files are a good start.

In the nearest week or two, considering I'll have enough time, I'll port all my PS2 tools to work properly with PC version. If you still need them by then, I'll try to post a package here, but it's always more fun if you do it on your own. At least that's how it works for me.

innocentius92
n00b
Posts: 14
Joined: Sat Dec 16, 2017 9:10 am
Has thanked: 13 times

Re: Okami HD PC - Texts files

Post by innocentius92 » Sat Dec 16, 2017 1:33 pm

ffgriever wrote:The first thing you need to realize is that Okami was initially a Wii (and then PS2) game. This means very strict memory and computing power constraints. That's one of many reasons why every format was developed to need the least possible memory footprint and the least possible processing. An XML parser or some other text parser could add a hundred or even few hundreds kB to the application size, then it would require a lot of memory for intermediate buffers before it's processed to the form that can be displayed directly. With small and simple games that's not an issue, but becomes one once you do something as big as one of the AAA games. That's why most games of that era (and before) opted mostly for relative pointers (which sometimes would be changed on the fly to absolute memory addresses), character substitution (containing only the used character glyphs and assigning them numbers in order of appearance in font table or font image), bitmap fonts, etc. That meant you had to place a file in memory only once, sometimes do some minimal processing (like pointer recalculation) and then use the values as they are, without any intermediates.

"Take a value CHAR from text file, take values from font description file at position CHAR*DESC_SIZE, use the values to determine X, Y, W, H coordinates of the glyphs to display, print a glyph from the font texture with given coordinates". That's very simplistic description of what usually happens in PSX/PS2 games (was a little different in nes/snes era, due to tiles, memory addressing, etc.).

With that mindset it will be much easier to take on the Okami file formats.

Anyway, to the point. ~90% of text in Okami is contained within MSD files, which in turn are contained in binary containers alongside many other files. The binary containers are usually one per location/menu/etc. For example st1/r120.bin is an intro stage. You know, "A long time ago in a galaxy far, far away..." or something along the lines, it's been a while. The file formats are VERY straightforward, probably among the most simple ones I've ever found in a game.

These are very similar to PS2 files, with one exception being that at pointer-0x20 location there is a metric with file type and stage name which wasn't present in PS2 version. Which in turn makes all the actuall file sizes next_pointer-previous_pointer-0x20 (except for the very last one, which just spans to the end). From the game perspective it doesn't matter at all (remember, it just loads it and uses the data, it doesn't dissect it into files).

So the file starts with 32bit number of files within container. Then a 32bit pointers to all the files, followed by 32bit magic strings describing file formats. What's interesting for you is the MSD file (or files).

As above, it starts with number of dialogs within a file (doesn't mean number of windows, but full dialogs, sometimes with multiple characters). Then come pointers. Each pointer is either 64bit or 32bit followed by an unknown/reseved 32bit value (I don't recall ever seeing it used). Then at each pointer relative to file start a dialog starts. Each character (or command value) consists of 16bit (two bytes). You'll need to assign characters to the values, but eg. 0xA301 (or 0x01 0xA3) is an action of waiting for gamepad/keyboard entry to move the dialog forward, 0xA101 is a "clearscreen" command, etc.

If you're about to edit Japanese files, please note, that there are also special commands to enable furigana and the text will be intertwined with furigana text. In Japanese version there will also be a lot more images in the files. That's because main Japanese font contains only numbers, romaji, kanas, few hundred most popular kanjis and some other characters. If a particular stage file uses kanjis/other characters not present in the main font file, the font pages are added to the stage file and accessed through special character range, which essentially means that for different stages a particular 16bit value may actually mean other characters. There are few cases when this is used for English version, too.

Some text is contained within other binary files with other formats, on PS2 a few labels were also hardcoded into executable, but the MSD files are a good start.

In the nearest week or two, considering I'll have enough time, I'll port all my PS2 tools to work properly with PC version. If you still need them by then, I'll try to post a package here, but it's always more fun if you do it on your own. At least that's how it works for me.
Ok, i will wait. But please, dont forget me. I really need this.

Zolodei
n00b
Posts: 10
Joined: Sun Jun 18, 2017 7:15 pm
Been thanked: 4 times

Re: Okami HD PC - Texts files

Post by Zolodei » Mon Dec 18, 2017 9:57 pm

Could, check your program can pull out the text, PC version?
https://cloud.mail.ru/public/6ecc/QZ42MTxh5

innocentius92
n00b
Posts: 14
Joined: Sat Dec 16, 2017 9:10 am
Has thanked: 13 times

Re: Okami HD PC - Texts files

Post by innocentius92 » Tue Dec 19, 2017 9:46 am

Zolodei wrote:Could, check your program can pull out the text, PC version?
https://cloud.mail.ru/public/6ecc/QZ42MTxh5
Yes, PC version.
I managed to visualize texts of the game using a thingy table, from your file Core_20_dec_0.rar, that could be modified with TranslHextion.
How could I correctly extract the files from Core_20.bin? There is text within the MSD file that could be modified for translation.
Image

Now I am downloading Okami HD PS3 version to compare the files.

innocentius92
n00b
Posts: 14
Joined: Sat Dec 16, 2017 9:10 am
Has thanked: 13 times

Re: Okami HD PC - Texts files

Post by innocentius92 » Tue Dec 19, 2017 11:11 am

I am thinking of some tool that facilitates the reading of the MSD files.

Zolodei
n00b
Posts: 10
Joined: Sun Jun 18, 2017 7:15 pm
Been thanked: 4 times

Re: Okami HD PC - Texts files

Post by Zolodei » Tue Dec 19, 2017 3:32 pm

innocentius92, give the table for MSD to write a program, and you know the structure of the MSA file?

innocentius92
n00b
Posts: 14
Joined: Sat Dec 16, 2017 9:10 am
Has thanked: 13 times

Re: Okami HD PC - Texts files

Post by innocentius92 » Tue Dec 19, 2017 3:49 pm

Zolodei wrote:innocentius92, give the table for MSD to write a program, and you know the structure of the MSA file?
I don't know much about this, but, I give you the table, i need learn more about this. And now I will patiently wait for your results. You have helped me a lot.

Okami tbl
https://mega.nz/#!pgATlIrY!CA_D_8uWDZhE ... Jv2DZIcd7I

keep me informed please.

Zolodei
n00b
Posts: 10
Joined: Sun Jun 18, 2017 7:15 pm
Been thanked: 4 times

Re: Okami HD PC - Texts files

Post by Zolodei » Tue Dec 19, 2017 6:20 pm

Sample text https://cloud.mail.ru/public/HQSw/UzfjgP956

innocentius92, ffgriever can share the file structure *.MSA, or describe the algorithm?

innocentius92
n00b
Posts: 14
Joined: Sat Dec 16, 2017 9:10 am
Has thanked: 13 times

Re: Okami HD PC - Texts files

Post by innocentius92 » Tue Dec 19, 2017 7:25 pm

Zolodei Can you say me how can I extract MSD and MSA from Core_20.bin please?

Zolodei
n00b
Posts: 10
Joined: Sun Jun 18, 2017 7:15 pm
Been thanked: 4 times

Re: Okami HD PC - Texts files

Post by Zolodei » Wed Dec 20, 2017 12:05 am

On writing a program to extract, but it is not ready yet, can not package files.
I want to add support for text and font.

ffgriever
beginner
Posts: 28
Joined: Fri Dec 15, 2017 5:21 pm
Has thanked: 1 time
Been thanked: 27 times

Re: Okami HD PC - Texts files

Post by ffgriever » Thu Dec 21, 2017 11:31 am

Quick update. I've written a simple unpacker/repacker. It can handle most of the binary containers, while repacking it also handles 64bit runtime offset file if it's present (it essentially contains the same pointers that at the beginning, only 64bit length and it's overwritten in memory to actual runtime offsets).

I've also fixed the descriptions in decrypter/crypter to match their real function.

Here's the link: http://ff12.pl/down/okami-tools-20171221.zip

I've tried it with few files edited and it seems to be working, but I did not do any extensive tests. Note, that it might produce smaller files (original files seem to have some additional 16byte padding here and there, but it's not needed as long as the files start at 16byte boundary).

Initially I was going to rewrite my PS2 multi tool/editor/viewer, but that's just too much hassle. It's much faster to just take care of the files with CLI. GUI was useful for the original translation, but is a waste of time for porting.

Image

As for MSA files. I should have my notes about it somewhere. I've never written any tool for that as I just had to add ęóąśłżźćńĘÓĄŚŁŻŹĆŃ and for that I could just copy both the glyphs (edit them a little) as well as their font table data (with some minor changes to improve kerning - there is no true kerning possible though, just some basics). I'll post the notes if I'm able to find them... Or reverse the format again. I guess you can't avoid editing it properly for a completely different script (Cyrillic in your case).

As for MSD extractor/repacker, I should be able to handle that on Saturday. The PS2 code should be fully reusable.

innocentius92
n00b
Posts: 14
Joined: Sat Dec 16, 2017 9:10 am
Has thanked: 13 times

Re: Okami HD PC - Texts files

Post by innocentius92 » Thu Dec 21, 2017 3:23 pm

ffgriever Can you share with us the tool you use in your screenshot?

User avatar
makcar
veteran
Posts: 152
Joined: Mon May 12, 2014 10:41 pm
Has thanked: 22 times
Been thanked: 45 times

Re: Okami HD PC - Texts files

Post by makcar » Thu Dec 21, 2017 4:01 pm

Maybe filelist_dat.lst, filelist_bin.lst etc to packing all?
Image

Code: Select all

for /R %%a in (*.ddp;*.tbl;*.emd;*.eff;*.tbl;*.scp;*.akt;*.cmp;*.pac;*.idd;*.bin;*.dat) do okami-pack.exe -p "%%a" "%%~dpna"
How about .emd and this archive https://mega.nz/#!4a5mCA5S!KlCTKDh3i3QG ... m1fB3mr2ec?
You have src?

innocentius92
n00b
Posts: 14
Joined: Sat Dec 16, 2017 9:10 am
Has thanked: 13 times

Re: Okami HD PC - Texts files

Post by innocentius92 » Sat Dec 23, 2017 3:20 am

ffgriever Initially I was going to rewrite my PS2 multi tool/editor/viewer, but that's just too much hassle. It's much faster to just take care of the files with CLI. GUI was useful for the original translation, but is a waste of time for porting.
Image
Can you share with me your Translation Tool and PS2 multi tool/editor/viewer, I need to do some tests with the original PS2 iso and its MSD files. I would really appreciate it. I love this game very much and I would like to be able to do something to bring the PC port in Spanish to the community.

Post Reply