Crash Bandicoot 1/2/3 (PSX) - NSF/NSD File Archives

[chekwob]

EDIT: @ chekwob, are you able to do working extractor/importer?

Extraction is possible, yes, but there's no filenames.
Importing is a bit more involved. Aside from messing with a few boxes in a hex editor, I haven't actually tried modifying the files yet. Replacing already-existing data might be easily done, but I can't really guarantee that.

Another thing that worries me is this, from the "Crash Bandicoot 2 Game Mod: Master Quest" blog:

All of Crash 2's NSF files have a checksum on them, and changing any data means the checksum needs recalculating. Chadderz, my younger brother, was the one who cracked the checksum; the program updates it upon saving. The "Rehash a file" button has the same functionality, only you don't have to load the file into the editor if you use that button. It's useful if you've been messing with things in a hex editor and want to quickly get things tested.

I haven't run into any kind of checksum that I'm aware of. Moving boxes around at the start of Snow Go didn't seem to result in any kind of errors when testing in ePSXe. Is the checksum a PAL-specific thing (I have the NTSC-U version)? Is my emulator setup somehow bypassing this checksum?

On a side note, my hex editing of the boxes was done on a raw ".bin" image of the disc. These things have an error detection and correction system. A small alteration like changing a box to a nitro should have been corrected, but it wasn't. Does ePSXe not do error correction? If not, does the actual PSX do it?

[ikskoks]

Is the checksum a PAL-specific thing (I have the NTSC-U version)? Is my emulator setup somehow bypassing this checksum?

On a side note, my hex editing of the boxes was done on a raw ".bin" image of the disc. These things have an error detection and correction system. A small alteration like changing a box to a nitro should have been corrected, but it wasn't. Does ePSXe not do error correction? If not, does the actual PSX do it?

Yes, checksum is a PAL-specifing thing, but i can work on NTSC-u version too. It doesn't matter.
Also libcrypt is specific for PAL games.

From what i know ePSXe has error correction (ECC/EDC). PSX doesn't have that function and games with errors can't launch at all.

EDIT: @ chekwob, i am waiting with pleasure for your great utility

[chekwob]

Here's a compiled version of my program so far (.NET 2.0):

https://www.dropbox.com/sh/yv93g4wsdde32s3/NuCSJR37Oo

It can open NSF files and extract individual items from it, but nothing more. This isn't really useful for any modding purposes yet; my focus right now is on reverse engineering.

That aside, here's a general overview of the NSF file format: (EDIT: I've updated this list a bit)
The NSF files are containers for "chunks", which are containers for "entries", which are containers for actual strings of data (often one string per entry, but not always). These terms are made up and don't appear anywhere on the actual game disc. There are 5 types of chunks:

• T0 - The most common type. Any entries which don't belong in another chunk type end up in these.
• T3 - Container for T12 entries (sound effects).
• T4 - Container for T14 entries (music instruments).
• T5 - Container for T20 entries (speech).
• T1 - Not actually an entry container. These contain textures and have a completely different format from the other chunk types.

There are 13 types of entries:

• T1 - ???
• T2 - ???
• T3 - ???
• T4 - ???
• T7 - Entities
• T11 - ???
• T12 - Sound effects
• T13 - Music (VH & SEP)
• T14 - Music (VB)
• T15 - ???
• T19 - Demos
• T20 - Speech
• T21 - ???

I've figured out a little bit of the T7 entry format, but it's not enough to do any serious modification. The sound effects and speech are ps1 adpcm format. The music appears to be standard playstation VAB and SEP. I was able to guess the purpose of the T19 entries based on which files they appear in and how many times they do so.

[Friedslick6]

Here's a compiled version of my program so far (.NET 2.0):

https://github.com/ughman/CrashEdit/downloads

It can open NSF files and extract individual items from it, but nothing more. This isn't really useful for any modding purposes yet; my focus right now is on reverse engineering.

Oh, that's great development! Have you notified MrBean35000vr about your progress yet? He would probably be interested.

[chekwob]

Some more information:

• The crash 1 nsf files seem to have a significantly different format.
• I'm almost certain the T12 entries are VAG files without the header. It is also likely that T14 entries are "VAD" (edit: I meant VAB) files without the header.
• There's actually T17 entries as well. They only appear in crash 3, and only in a few levels.
• T11 entries sometimes have string constants in them, such as "SEWER OR LATER" or "BONUS". My guess is these are related to GOOL, the LISP dialect used in the games.

I've been working on adding a function to extract T12 entries as WAV files, but this sound format is balls, and there's not much available information on it.

Have you notified MrBean35000vr about your progress yet? He would probably be interested.

I have not. I'm not sure what I could even say.

[Friedslick6]

Hmm, I think I'll just post him a comment about this topic then...

[chekwob]

Another version available (v0.02):
https://www.dropbox.com/sh/yv93g4wsdde32s3/NuCSJR37Oo

New features:

• Playing sounds/speech and extracting them as wave files. Only 11025 Hz and 22050 Hz are supported. Some sound effects use neither of these, but most speech does.
• Texture chunks are displayed as 8bpp grayscale and 16bpp BGR555 graphics.
• Entity entries are partially parsed now.
• Various other minor changes.

It's still not extraordinarily useful for modding, but it's progress. Crash 1 archives are still unsupported.

A list of the levels and their associated filenames can be found at https://sites.google.com/site/crashpsxsite/structure. The list misses 3A for crash 3, which is the vortexy area where the enemy bosses talk to you. (EDIT: nevermind, it has that now too)

[chekwob]

I don't want this to become an announcement thread for my program, but I feel that some people might be particularly interested in this release.

New download link for v0.04:
https://www.dropbox.com/sh/yv93g4wsdde32s3/NuCSJR37Oo
(github has deprecated the downloads page, so I'm using dropbox now)

This version features extracting music in SEQ or MIDI format. I've made a post explaining how to do this here.

[ikskoks]

some people might be particularly interested in this release.

I'm very interested

[chekwob]

I've been creating some new documentation, this time with diagrams and stuff.
https://dl.dropbox.com/s/fu29g6xn97sa4p ... ormat.html
This is most of what I know, but it is missing a few details. This document also includes details on the common playstation formats SEQ, SEP, VAB, VH, and VB.

Something I noticed a few weeks ago is that I can't actually modify the PAL game files. Loading and saving them works fine, but the PAL game engine rejects any changes. I've spoken with MrBean, and learned that there are indeed checksums in the NSF files. Every 64kb section in the file has a checksum, and if the checksum doesn't match up with the rest of the data, the game rejects the data as invalid. MrBean's master quest blog post said as much, but I had no idea what he was talking about at the time because, as it turns out, the NTSC version of the game engine doesn't care about the checksums. Why it doesn't care is a mystery, but it's thanks to that that I was able to get this far. This puts a stop to modifying the PAL version until I can acquire more details on the checksum.

I've made some progress on modifying the in-game demos that play when you idle at the title screen. There's some weird incremental values in there before each set of inputs, probably some kind of timing information to ensure they don't desync.

MrBean found where the 3d models are inside the files, but nobody really knows the format, so ripping and modifying them is not possible at the moment.

Most of the music-related stuff is well known now. I'm trying to implement exporting DLS files so the extracted MIDIs can be played back and sound (mostly) like they do in-game. Unfortunately, I have no idea what I'm doing with this DLS stuff. I may have to read the entire DLS spec from front to back to figure it out.

On another note, there was a beta version of Crash 2 PAL released recently. I've confirmed that it works with my code, but unfortunately it also has the checksum limitation of the retail PAL version. This checksum thing always seems to be PAL-specific. Even the retail crash 3 ntsc doesn't care about the checksum. Maybe the checksum was something SCEE wanted?

Also, still no progress on Crash 1. My current theory is that Crash 1 NSF files are compressed in some way. I haven't really paid much attention to Crash 1, sorry.

[ikskoks]

Good news. I cross my fingers for your work

This checksum thing always seems to be PAL-specific.

Yeah, the same situation was in MediEvil PAL. Guys from Russia solved that problem by deleting some files from game image and creating new iso ;p

[chekwob]

I've made two significant breakthroughs since my last post.

The first is the checksum. Someone sent me an email with some very useful information on how the checksum is calculated in the crash games. Thanks to that, I was able to write an implementation in C#, and CrashEdit can now modify the PAL versions.

The second is Crash 1's NSF files. I had previously assumed that Crash 1's NSF files would be left a mystery for a long time due to lack of interest, but today I took a crack at disassembling the Crash 1 EXE. I've made a surprising amount of progress, and I believe I can implement loading of Crash 1 NSF's into CrashEdit soon.

Also, if any of you have a real, non-downloaded copy of Crash 2 PAL, could you please post the md5 hashes of the following files:

• S2/S000002D.NSF
• S2/S000002E.NSF
• S2/S000002F.NSF

Thanks.

[DCxDemo]

you are doing a great job dude! adding this thread to bookmarks.

[chekwob]

I have identified the models for game objects as entries with type 3. (EDIT: I meant type 2, why do I keep mixing these numbers up) I'm still working on figuring out the format. Most likely, color will be figured out first (as it appears to be fairly simple), then texture mapping, and then eventually the actual vertices of the model. Animation data may or may not be separate from the models themselves, I'm not sure yet but I suspect that it is.

[Friedslick6]

Animation data didn't appear to be separated from the model when I tried forcing a bind pose on Crash Bandicoot's model, but we'll see.
I can't be of much use except in telling you what I already know and haven't posted in this thread already (regardless of redundancy lol.)

• The colours are applied through a gouraud shading method.
• Crash's eyebrows are grey-scale and the assigned polygons are coloured separately.
• When Crash's back texture was ripped through an emulator using ePSXe with the Next 3D 1.5 Video plugin and through 3D Ripper DX, the textures were stacked in an unusual way:
  Segmented into 32*32 sized tiles, with each top-right tile moved below the top-left tile, and each bottom-left tile moved above the bottom-right tile within each group of 2*2 tiles.
  (Other ripped textures did not share this oddity.)

Good luck to you.