Reflections CHUNK File Format

[Chipicao]

Well I have no idea what he has access to and what not :p

But thanks, I asked because I've been researching DSF for some time and the format is similar except for some chunk names.

[WRS]

But thanks, I asked because I've been researching DSF for some time and the format is similar except for some chunk names.

what research can you share with us on dsf? have you looked at the model data?

also for the moment i'm just curious what driv3r does with its data as i used to play the xbox version and really enjoyed it but it sounds like there are enough similarities to make something more generic if possible

[Chipicao]

I'm not sure about Driv3r (didn't have time to analyze your research thoroughly, sorry) but I'll tell you what I know about DSF .sp files
- every file is split into one or more chunks, identified by a header name called CHNK
- first long is the total chunk size starting from the beginning of CHNK
- second long is the number of sub-sections (sub-chunks)
- third long is the number of parameters (longs) for each sub-section; there are usually 3, but you should always keep count of this number.

The sub-sections start immediately after this. They are actually sub-section headers; their actual data starts after every sub-section header is parsed, and is identified with offsets.
- every sub-section header starts with a 4-character name (example: MDXN - model data, NONR - "non-resources" like car sounds)
- the first parameter is an offset to the sub-section data/contents; this offset is relative to the beginning of the "parent" CHNK header
- the second parameter is unknown at this time
- the third parameter is the sub-section data size, starting from where the offset takes you

The contents of each sub-section may be actual data, or it may be another CHNK, in which case the above applies all over again.

The model data is in specific section names, for example DXTC for DDS textures, or VXDT for vertex data.

[CarLuver69]

- third long is the number of parameters (longs) for each sub-section; there are usually 3, but you should always keep count of this number.

Ah, so that explains the mysterious long we've been trying to figure out. WRS thought it was a version number, I thought it was just unknown, but thanks for clearing that up. The formats are 100% similar.

[Axsis]

I did some research in D:SF too. About a year ago... so I don't remember any details about format, but then I've wrote a script for chunks unpacking. Not sure if it would work with DRIV3R, but if you think that formats are similar you may try it.
As I remember, it unpacks all chunks recursively, as result giving LOTS of small files in LOTS of folders (messy...) But maybe it'll be useful for someone...
Good luck in further research!

Code: Select all

Set FULLNAME string ""
Get NAME basename
CallFunction ChunkParser

################################
StartFunction ChunkParser
 String FULLNAME += \
 String FULLNAME += NAME
 GetDString HDR 4
 If HDR != "CHNK"
  String FULLNAME += ".bin"
  Log FULLNAME RECOFFSET RECSIZE
 Else
  SavePos CHNKOFFSET
  Math CHNKOFFSET -= 4
  Get CHNKSIZE long
  Get NUMREC long
  Get DUMMY long
  For REC = 0 < NUMREC
   GetDString RECID 4
   Get RECOFFSET long
   Math RECOFFSET += CHNKOFFSET
   Get DUMMY byte
   Get RECNAMESIZE byte
   Get DUMMY short
   Get RECSIZE long
   SavePos POS1
   Math RECNAMEOFFSET = RECOFFSET
   Math RECNAMEOFFSET += RECSIZE
   GoTo RECNAMEOFFSET
   GetDString RECNAME RECNAMESIZE
   String NAME p= "%04u_%s_[%s]" REC RECID RECNAME
   Goto RECOFFSET
   CallFunction ChunkParser
   GoTo POS1
  Next REC
 EndIf
EndFunction
################################

[WRS]

its interested that a few people have managed the chunk parsing but nobody has gotten much further

[CarLuver69]

The easiest part is "unchunking" the files. The hardest part is actually making use of the files inside the chunks, which tend to be formats none of us have experience with.

[Antares666]

Hi, this topic is quite interesting, as a driv3r fan, i am following it with great interest. Any progress since your last post ?

However, WRS, did you use a max script to visualize the normal data ? If so, would you be willing to share the script please ? Because i would like to see the normal datas in Max ( i have tested your tool Explor3r, nice work ). Thank you

By the way, good job guys.

minor update

slow progress with the model data

the renderer model package chunks (MDPC) contain the models (faces indices, vertex data and textures) but i'm stuck on reconstructing them. all i think i've found is the normal data:

[WRS]

when i last worked on this i was looking for face indexes

instead of the 3d wheel rims i was looking at the maps (which are stored with 3d geometry but are 2d). i started by using 3d ripper dx to save the meshes in the scene, then deleting all but the map.

obviously the vertex info isn't the same due to projects and whatever, but i got nowhere looking for face details.

the maxscript was written on the fly. i found a chunk of data which looked like a vertex buffer (the 32-bit values were all valid floats) and created a rough mesh based on it.

i then decided got distracted writing a 3d model data explorer: https://github.com/x1nixmzeng/luaexpose

i have since been distracted by something else

edit, i've attached the map i mentioned

[RacingFreak]

Holy crap! Fantastic

[CarLuver69]

I think what you seek is not a face index, but an indice index. Try looking in the .MAP files at offset 0x3360, there's a common header of

Code: Select all

01 00 00 00 A0 86 01 00

which seems to be a header, further down a little there's a bunch of indicies.

I think I found some kind of index, it's just before the indices I talk about:

[WRS]

wip 010 binary template for mdpc chunks (explor3r can save these for you)

edit
this clearly wasn't helpful. if anyone needs it, pm me

[CarLuver69]

That template is pretty useful for the most part. However I think it would be better to just document the actual MDPC format rather than make an 010 template, since from what I can see, it doesn't do much good other than allow you to visually jump to each section in the header. It also skipped a section of bytes near the "partA" section and messed up the offsets. It's pretty cool otherwise.

[WRS]

That template is pretty useful for the most part. However I think it would be better to just document the actual MDPC format rather than make an 010 template, since from what I can see, it doesn't do much good other than allow you to visually jump to each section in the header. It also skipped a section of bytes near the "partA" section and messed up the offsets. It's pretty cool otherwise.

i have no idea what the actual MDPC format looks like.. this template is the result of guesswork working from a chunk i dumped. what do you mean "document the actual MDPC format"?

[CarLuver69]

You made some pretty accurate guesses on the MDPC format, so instead of making a 010 template, take notes on the format and post them here kinda like how I did when I first started the thread. I made a pretty cool discovery regarding vehicle hierarchies, here's the result of messing around -









The method I used was the equivalent of swapping vehicle IDs. I documented every vehicle ID in the game -

Code: Select all

ID		CITY		NAME
				
0x02	Miami		Flamingo Taxi
0x06	Miami		'69 Bruiser
0x09	Miami		Police Prowler
0x0A	Miami		El Toro GT500
0x13	All			Packer Big Daddy
0x14	All			Packer Big Daddy (Trailer)
0x16	Miami		Dart Retaliator
0x17	Miami		Scout Cargo Van
0x18	Miami		G-750 Pickup
0x1F	Miami		Antilli VO9s
0x20	Miami		Scout Chaser
0x21	Miami		Hunter 313-T
0x22	Miami		V-8
0x23	Miami		Patriarch Tycoon
0x24	Miami		Packer Transport (Dump Truck)
0x25	Miami		'80 Redline V-8
0x26	Miami		Grande Spedizione
0x27	Miami		Miami Bus
0x28	Miami		Adams Liberty
0x2B	Miami		Go-kart
0x2C	Miami		Farley FLH Whole Hog
0x2D	Miami		Cigarette 38 Top Gun
0x2E	Miami		Sea-King Cormorant
0x2F	Miami		Surf Craft
0x31	Miami		Monorail
0x40	Miami		Sobe Packer Truck
0x42	Miami		Velocity Turbo
0x50	Miami		TT Cuatro Superpower
0x59	Miami		Miami Dade Police Boat
		
0x0D	Nice		Benissimo 74 Turbo
0x10	Nice		Prontezza Freddo
0x11	Nice		Le Compact Rapporter
0x19	Nice		Vitesse Moyenne 72X
0x1B	Nice		Dolch Schub
0x2A	Nice		Dagger Type-S
0x32	Nice		Le Compact Taxicab
0x33	Nice		Dagger Type-T
0x34	Nice		Le Chariot Transport 6
0x35	Nice		Le Chariot Klein
0x36	Nice		Forklift
0x37	Nice		Conquest Motors Dominance
0x39	Nice		Vitesse Moyenne 94 Police
0x3A	Nice		Le Autobus
0x3B	Nice		Lastwagon Kasten
0x3C	Nice		Dolva 8M8
0x3D	Nice		LTS V-8
0x3E	Nice		Le Chariot Cinq
0x41	Nice		Camper Van
0x43	Nice		Sun Runner
0x44	Nice		Moped
0x45	Nice		Cigarette 42 Tiger
0x46	Nice		Sea-King Cormorant
0x47	Nice		Sea-King Silverfish
0x5A	Nice		Nice Polis Boat
0x5C	Nice		Vitesse Moyenne 94LE
0x5D	Nice		Le Compact XS
0x5E	Nice		Prontezza Brezza
		
0x0E	Istanbul	Le Chariot Douze Polis
0x0F	Istanbul	Cargo Van
0x15	Istanbul	'54 Taxi
0x1C	Istanbul	Santun TTZ
0x1D	Istanbul	'54 Classic
0x1E	Istanbul	'71 Pickup
0x29	Istanbul	Speedster
0x48	Istanbul	'73 Classic
0x49	Istanbul	Otobus
0x4A	Istanbul	Le Chariot A1
0x4B	Istanbul	Canyon Wagonaire
0x4C	Istanbul	Packer Transport (Flatbed)
0x4D	Istanbul	Jager Roadster LS28
0x4E	Istanbul	Roadster
0x4F	Istanbul	Racer GT
0x51	Istanbul	Beast
0x52	Istanbul	Moped
0x53	Istanbul	Yeni Golata
0x54	Istanbul	St. Michael Mariner
0x55	Istanbul	Sport Fisher
0x56	Istanbul	Tram
0x57	Istanbul	Train Engine
0x5B	Istanbul	Istanbul Polis Boat
0x5F	Istanbul	Train car

Doing this allowed me to finally figure out how to locate vehicle-specific handling in BO3 files using the big header up top, the hex number represents the position in BO3 header so it can be assigned proper handling and vehicle colors. I've decoded BO3 file format 50%, the biggest part of it was finding car colors (they're just little-endian hexidecimal). Now to figure out what everything else does, which seems to be mostly performance related. I cannot confirm or deny if the BO3 files control wheel position since swapping vehicle IDs resulted in proper wheel placement, so I am thinking wheel placement is in AWHF format.

I'll try to get some documentation up soon.