XeNTaX

Posted: **Wed Nov 21, 2012 5:28 am**

Since DRIV3R, Reflections has used a form of this "CHUNK PACKAGE" file format that is basically a unified package containing everything from models to textures, positions in the game world, flags...everything you could possibly think of. It is important to note that the content inside these archives (if they are even archives) are not compressed in any way, shape, or form - one of the major aspects of these files is that there are offsets EVERYWHERE. The very beginning of the files define offsets and block size - there are even sub-chunks inside the big chunk that have offsets as well. Essentially what's going on is each chunk is like a tupperware container with smaller containers inside of it. Once the chunk format is decoded in DRIV3R, we can move onto other games such as DPL or even DSF! The format is exactly the same, the only differences are the new file formats inside of these chunks. Not a big deal, but having chunks out of the way would be nice!

There is no one specific file extension, as it varies depending on what it does. My research here is based on DRIV3R for the PC, and I essentially have no experience in programming myself, which is why I've taken up this project!

Here's some decoding of the CHUNK based file format, using MIAMI_DAY_PC/SuperRegions.pcs as an example:

NOTE: This image has outdated information. To be used for reference only!

CHUNK files have "PCMP" sections that hold DDS textures. It contains complex offsets (to me, at least) describing where each new DDS begins and how long it is. A good friend of mine helped me decode the headers a bit, here's what we came up with (sorry if it's not very specific, I didn't make this picture but will when further documentation arises):

UPDATE 11/22/2012: New organization, more info added.

Posted: **Wed Nov 21, 2012 8:56 am**

This post is reserved for an in-depth analysis of each file.

CITIES
The cities of DRIV3R is basically what makes up the whole game - Miami is 390MB, Istanbul is 319MB, and Nice is 286MB. On average, each city has close to over 3,500 textures and an unknown amount of models. According to older work-in-progress screenshots, a program called "Mr. RoadBuilder" was used to create every aspect of the city itself. Here are the only screenshots ever released:

This could give potential reverse engineers an insight into how everything works. Note in the upper-right corner there are references to "Global Regions" and "Super Regions" - these are important to take note of, as they play a crucial part in my observations below. Without further ado...

Legend:

-- = Folder
---- = Inside Folder

city = Name of the city (Miami, Nice, or Istanbul)
timeofday = DAY or NIGHT
platform = I'm assuming what platform it's mean for (in this case, PC = _PC)

.d3c = DRIV3R City. These are known as "Global Regions" that basically make up the whole of the city. The .D3C file interacts with the .PCS file somehow, as there are no models or textures inside.
.pcs = Models and textures for the city. How the .D3C files access these is currently unknown.

--city_timeofday_platform*
----Interiors.pcs**
----SuperRegions.pcs***
city_timeofday_platform.d3c****

* - By default there are 6 folders - Miami with day textures is called "MIAMI_DAY_PC" and with night textures is "MIAMI_NIGHT_PC". The rest is self-explanatory.
** - Definitions for interiors are in here.
*** - Definitions for the outside world you drive in.
**** - For each folder, day or night, there is a .D3C file accompanied by it.

.PCS Files - Last Updated 11/22/2012
.PCS files contain the models and textures that make up the city. The .D3C file uses this file as an index in order to retrieve models amongst other things.

.D3C Files - Last Updated 11/22/2012
These files are very rich in content - these are the biggest CHUNK based files in the whole game. There's so much to a city it's unbelievable. More information will be put here later on.

CONFIGS
Vehicle configurations for every vehicle in the game are in here.

----Vehicles
----BigVO3
----*.b03*

* one for each city

.B03 Files - Last Updated 11/22/2012
While they're not CHUNK based files, they are still a crucial part of the game that need to be documented more. Vehicle handling, colors, etc. for every vehicle in the game.

GUNS
Weapons data located in here.

--Guns.cpr

.CPR Files - Last Updated 11/22/2012
Guess who forgot to document these files?

MISSIONS
The mission scripting language related files can be found here.

--Personalities
----*.acp
--Recordings
----*.pad
*.dam
*.mpc

.ACP Files - Last Updated 11/22/2012
AI behavior files. Not much is known.

.PAD Files - Last Updated 11/22/2012
Vehicle recording files. These were probably made in a special developer mode that records player movement similar to the Film Director.

.MCP Files - Last Updated 11/22/2012
Exported mission scripts. There was most likely a stage where they were written in a plain-text format and later converted to this special format that uses IDs from locale files.

.DAM Files - Last Updated 11/22/2012
Contains models and textures for each mission. There are a variety of things in here, so its main purpose is unknown. However, it is used to define character models/textures/joints/etc. !

OVERLAYS
I'll update this section at another time.

*.bin
*.gfx
*.map

SFX (Sound Effects)
I'll update this section at another time.

Sfx.pmu

Skies
I'll update this section at another time.

*.d3s

Sounds
I'll update this section at another time.

*.dat
*.vsb

GUI
I'll update this section at another time.

*.mec

Vehicles
I'll update this section at another time.

--city
----CarGlobals[city].vgt

*.VVS
*.VVV

Posted: **Wed Nov 21, 2012 9:39 pm**

i found the demo online. there is a description offset in the headers which is most useful .

other notes i made:

Data is chunked with the following header:

uint32 magic ("CHNK")
uint32 fileSize
uint32 chunkCount
uint32 version (3 == Driv3r?)

The minimum size is 16 bytes

Chunks have the following header:

uint32 magic
uint32 chunkOffset
uint32 flags (? based on what you found out)
uint32 descOffset

chunkOffset is relative to the start of the CHNK header
descOffset is relative to the chunkOffset and CHNK header

Chunk data is aligned to 4096 bytes
So are descriptions

Empty memory is filled using the magic DEC015A1h

This is useful when reading the description info:

You can read 4 bytes at a time and xor until you hit 0

Other

Chunks may contain more chunked data (with CHNK magic)

GESR Super region
GEGR Global region
GEIR Interior region
PINF Mission editor model info
PCSL Dummy chunk
MDPC Model package

Files using this structure:

Overlays/*.map
Overlaps/*.gfx
Cities/*.d3c
Cities/ (see op) /*.pcs

Posted: **Thu Nov 22, 2012 4:34 am**

UPDATE 11/22/2012:

Lots of new info today. Wrote a small BMS script (see below) and took the time to document every single file type in the game. Without further ado, here are my huge ass research documents!

I'm not quite sure what I was going for with this one, but it seems to be useless since the one below this one is almost the exact same.

Code: Select all

CHNK (Reflections CHUNK Container)
	- Archive-like tendencies:
		- Defines size of chunk
		- Defines # of packages inside
	- Always has an unknown LONG at the end; "03 00 00 00" (3 in little endian)
		- Possibly version number (credits go to user WRS)
	- Package offsets begin on the line of the CHNK header:
		- For example, let's say "BLAH" has offset of "00 10 00 00" (or 1000). This means the beginning of "BLAH" is offset 1000 from the CHNK header. Got all that?
		- Offsets are 4096 byte-aligned (credits go to XeNTaX)
	- Files inside CHUNK's are uncompressed ASCII format, however they are not common file formats and are not meant to be used as external files. Files inside chunks need to be extracted, modified, and have offsets/checksums recalculated - a relatively easy process once everything is totally understood.
	- Acts as a container
		- Nesting capabilities:
	
			CHNK
			[parent]
				CHNK
				[child/parent]
					CHNK
					[child/parent]
						CHNK
						[child/parent]
						...
				[child]
				[child]
			[parent]
			
		- Think of the chunk as a big tupperware container; You can put other smaller containers inside, as well as a couple of other things on the side.

"Unified Package"
	- The result of a package having no specific name assigned (all 00's)
	- Acts as a container that holds another chunk inside of it, most likely so the main chunk itself can access it and then start doing sub-routines. Why this is necessary is unknown, maybe just for organization.
	- No further information.
	
-------------------------
%CITY%.D3C
------------------------- 
GESR (Global Region)
----GEPM (Building Physics Data)
----PMLD (Building Physics Lookup Data)
----GWTR (Water Instance Data)
----GEBI (Global Building Instance Data)
----GERD (Spline Road Data)
--------AIEX (AIExport)
----ANOD (Animated Object Data)
----SCND (Global Scene Data)
----GEPD (Prop Data Container)
--------HRDT (Prop Hiearchy Information)

GEIR (Region %0d)
----GWRD (Water Region Data)
----GEBI (Building Instance Data)
----GRTD (Region Terrain)
----GEBR (Building Routefind Data)
----JNR! (Broadphase Data)
----ATRD (Attractor Data)
----SCND (Region SceneData Data ?)
----OCGD (Occluder Game Data)

IRCT (External file link + External?)
----ATRD (Interior Attractor Object Data)
----ANOD (Interior Animated Object Data)
----GEBI (Interior Instance Data)
----IRDT (Interior Scene Data )
----GEPM (Interior Physics Model Data)
----PMLD (Interior Physics Lookup Data)
----JNR! (Interior Broadphase Data)
----ISHM (Interior Lighting Data)
--------PROBv1.0 (unknown format - unused?)
GESR (again?) (External file link + Internal?)
----GEPD (Prop Data Container)
--------HRDT (Internal Interior Prop Hierarchy Prop-Data Container)

PINF (Model Information for Mission Editor)
----PIND (Model Data for Mission Editor)
----PINS (Model Strings for Mission Editor)

GEGL (Grandiose Lookup?)
----GESI (Super Region Info)
----GILH (Lookup Header)
----GSIL (Super Region Lookup )
----GRIL (Region Lookup)
----IRLU (Interior Lookup Info)

PCSL (Specific Region? "NORTH DOWNTOWN")
----GEPM (Building Physics Data)
----PMLD (Building Lookup Physics Data)
----ANOD (Animated Object Data)
	
-------------------------
MISSION%0d.MPC
------------------------- 
"Exported Mission Root"
Description: Basically a "shrink wrap" for the exported mission data.

----EM__ (Exported Mission)
	Description: Holds all mission-related data. Nested multiple times (rare).
	
--------EMOB (Exported Mission Objects)
--------EMPR (Exported Mission Prop Handles)

--------LELD (Logic Export Data)
------------LESC (String Collection)
------------LESB (Sound Bank)

------------LEAC (Actors)
--------------------LEAD (Actor Definitions)
--------------------LEPR (Actor Properties Table)

------------LENC (Logic Nodes)
--------------------LEND (Logic Node Definitions Table)
--------------------LEPR (Logic Node Properties Table)

------------LEAS (Actor Set Table)
------------LEWC (Wire Collection)
------------LECO (Script Counter)

-------------------------
MISSION%0d.DAM
------------------------- 
"Unified Packager"
Description: Presumably stuff related to models and characters for each specific mission.

----MDPC (Renderer Model Package)
----UPXD (Extra Character Data Package)
----UPCB (Skelton Package)

EMMS (Mission Summary)
Description: Probably used for locale purposes.

-------------------------
%CITY%.MAP
------------------------- 
"Unified Packager"
Description: The HUD map.

----MDPC (Renderer Model Package)

-------------------------
*.GFX
-------------------------
"Unified Packager"
Description: Packaged graphics, each with a different purpose.

----UPST (Standalone Textures)
----MDPC (Renderer Model Package)

-------------------------
SFX.PMU
-------------------------
"Unified Packager"
Description: No idea what these are for...sound effects need textures?

----UPST (Standalone Textures)
----MDPC (Renderer Model Package)

-------------------------
*.MEC
-------------------------
RDMP (Reflections Menu Data Package)
Description: Contains menu data chunks

----RMDL (Reflections Menu Data Chunk)
----MDPC (Reflections Menu Material Chunk)

-------------------------
FONT.BNK
-------------------------
FONT (Font Container)
Description: Holds information regarding fonts for DRIV3R. Universal.

----UPST (Standalone Textures [lContainerPa?])
----MDPC (Renderer Model Package)
----FBNK (Font Data)

-------------------------
CARGLOBALS%CITY%.VGT
-------------------------
"Unified Packager"
Description: Contains global content used for vehicles, such as interiors, brakelights, etc.

----UPST (Standalone Textures)
----MDPC (Renderer Model Package)

-------------------------
%CITY%.VVS
-------------------------
"Individual Model"
Description: Each individual car model, along with textures, is contained in here. UPVH has an unknown LONG at the beginning in place of the usual magic numbers - this should be noted.

----UPVH (Vehicle Hiearchy)*
--------AWHF (Unknown Format)

MDPC (Renderer Model Package)*

-------------------------
%CITY%.VVV
%CITY%_ARTIC_TRUCK.VVV
%CITY%_BOAT.VVV
%CITY%_BIKE.VVV
MISSION%0d.VVV
-------------------------
"Unified Packager"
Description: This is an interesting one. There's a .VVV file for the city, the semi truck, boats, motorcycles, and individual missions. Could this be vehicle selection?

----UPVH (Vehicle Hiearchy)*
--------AWHF (Unknown Format)
----MDPC (Renderer Model Package)*

* means multiple instances

This one is similar to above, but it mimics the exact format of the files:

Code: Select all

/CITIES

	../[CITY]_[TIMEOFDAY]_[PLATFORM]
	*.PCS
	
	CHNK
	MDPC - ModelPackage_PC

*.D3C

CHNK
GESR - Global Region
	CHNK
	GEPM - Building Physics Data
	PMLD - Building Physics Lookup Data
	GWTR - Water Instance Data
	GEBI - Global Building Instance Data 
	GERD - Spline Road Data
		CHNK
		AIEX - AIExport
	ANOD - Animated Object Data
	SCND - Global Scene Data
	GEPD - Prop Data Container
		HRDT - Prop Hiearchy Information
GEIR - Region
	CHNK
	GWRD - Water Region Data
	GEBI - Building Instance Data
	GRTD - Region Terrain
	GEBR - Building Routefind Data
	JNR! - Broadphase Data
	ATRD - Attractor Data
	SCND - Region SceneData Data ?
	OCGD - Occluder Game Data
IRCT - External file link + External?
	CHNK
	ATRD - Interior Attractor Object Data
	ANOD - Interior Animated Object Data
	GEBI - Interior Instance Data
	IRDT - Interior Scene Data 
	GEPM - Interior Physics Model Data
	PMLD - Interior Physics Lookup Data
	JNR! - Interior Broadphase Data
	ISHM - Interior Lighting Data
		PROBv1.0 (unknown format - unused?)
GESR (again?) - External file link + Internal?
	CHNK
	GEPD - Prop Data Container
		CHNK
		HRDT - Internal Interior Prop Hierarchy Prop-Data Container
		
PINF - Model Information for Mission Editor
	CHNK
	PIND - Model Data for Mission Editor
	PINS - Model Strings for Mission Editor
GEGL - Grandiose Lookup?
	CHNK
	GESI - Super Region Info
	GILH - Lookup Header
	GSIL - Super Region Lookup 
	GRIL - Region Lookup
	IRLU - Interior Lookup Info
PCSL - "NORTH DOWNTOWN" ?
	CHNK
	GEPM - Building Physics Data
	PMLD - Building Lookup Physics Data
	ANOD - Animated Object Data

/MISSIONS
*.MPC

CHNK
[blank] - Exported Mission Root
	CHNK
	EM__ - Exported Mission
		CHNK
		EMOB - Exported Mission Objects
		EMPR - Exported Mission Prop Handles
		LELD - Logic Export Data
			CHNK
			LESC - String Collection
			LESB - Sound Bank
			LEAC - Actors
				CHNK
				LEAD - Actor Definitions
				LEPR - Actor Properties Table
			LENC - Logic Nodes
				CHNK
				LEND - Logic Node Definitions Table
				LEPR - Logic Node Properties Table
			LEAS - Actor Set Table
			LEWC - Wire Collection
			LECO - Script Counters
EMMS - Mission Summary
	
*.DAM

CHNK
[blank] - Unified Packager
	CHNK
	MDPC - Renderer Model Package
	UPXD - Extra Character Data Package
	UPCB - Skelton Package
	
/OVERLAYS
*.MAP

CHNK
[blank] - Unified Packager
	CHNK
	MDPC

*.GFX

CHNK
[blank] - Unified Packager
	CHNK
	UPST - Standalone Textures
	MDPC - Renderer Model Package
	
/SFX

*.PMU

CHNK
[blank] - Unified Packager
	UPST - Standalone Textures
	MDPC - Renderer Model Package
	
/TERRITORY/[LOCALE]

	../GUI
	*.MEC
	
	CHNK
	RDMP - Reflections Menu Data Package
		CHNK
		RMDL - Reflections Menu Data Chunk
		MDPC - Reflections Menu Material Chunk
	
	../LOCALE/[LANGUAGE]..

		../FONTS
		*.BNK
		
		CHNK
		FONT - Font Container
			CHNK
			UPST - Standalone Textures (lContainerPa?)
			MDPC - Renderer Model Package
			FBNK - Font Data
		
/VEHICLES

	../[CITYNAME]
	*.VGT
	
	CHNK
	[blank] - Unified Packager
		CHNK
		UPST - Standalone Textures
		MDPC - Renderer Model Package
	
*.VVS

CHNK
[UNKNOWN long] - Individual Model
	CHNK
	UPVH - Vehicle Hiearchy
		AWHF (Unknown Format)
MDPC - Renderer Model Package

*.VVV

CHNK
[blank] - Unified Packager
	CHNK
	UPVH - Vehicle Hiearchy
		AWHF (Unknown Format)
	MDPC - Renderer Model Package

Posted: **Thu Nov 22, 2012 4:35 am**

alright, my turn to bump

i've written a rough bms script to dump the dds textures from the GFX files (for now)

overlays.gfx - gui stuff; weapon highlights
loading.gfx - the spinning disk icon and the driv3r logo (attached)

Code: Select all

# Driv3r (PC) *.gfx
# v0.01 - rough recursive script based on demo filedata
# WRS (xentax.com)

set OFFSET long 0
callfunction ParseChunk 1
cleanexit


startfunction ParseChunk
  getdstring BLOCKTYPE 4
  get BLOCKSIZE long
  get CHUNKS long
  get VERSION long

  if BLOCKTYPE != "CHNK"
    print "Unknown chunk header"
    cleanexit
  endif

  if VERSION != 3
    print "Unknown version"
    cleanexit
  endif

  for CHUNK = 1 to CHUNKS
    getdstring CHUNKTYPE 4
    get CHUNKOFF long
    get CHUNKUNKNOWN long
    get CHUNKSIZE long

    if CHUNKTYPE = "MDPC"
      print "Found info header"
      math LOCALOFF = CHUNKOFF
      math LOCALOFF += OFFSET
      math LOCALOFF += 4096  ## unknown heading
      goto LOCALOFF
      idstring "PCMP"
      get VER long # 3 again
      get FILES long
      getdstring IGNORE 32
      get HEADERINFO long
      math HEADERINFO -= 48 # we read 4+4+4+32+4 bytes
      savepos POS
      math POS += HEADERINFO
      goto pos
      ## now we have our headers
      math LOCALOFF += 4096 # again, assumptions
      for F = 1 to FILES 
        get FUNKNOWN long # 01010101h (no idea)
        get FCHECKSUM long # CRC32 of the DDS data
        get FOFFSET long # relative to LOCALOFF for now
        get FSIZE long
        getdstring FIGNORED 16 # ignorable data

        math FOFFSET += LOCALOFF
        string FNAME p= "%i_%08X.dds" F FCHECKSUM
        log FNAME FOFFSET FSIZE

      next F
      print "Done!"
      cleanexit
    endif

  next CHUNK

  if CHUNKTYPE = "" and CHUNKS = 1
    math LOCALOFF = CHUNKOFF
    math LOCALOFF += OFFSET
    math OFFSET = LOCALOFF
    #save CURPOS
    goto OFFSET
    print "Next level down.."
    callfunction ParseChunk 1
  endif

endfunction

more to come

Posted: **Thu Nov 22, 2012 4:41 am**

I actually wasn't expecting anyone to try and help out, this is quite unexpected. Here's a BMS script of my own (my very first one, I'm working on a more advanced one)

Code: Select all

# Reflections CHUNK File Format Scanner V0.1.1121.12 alpha
# © CarLuver69
# This code is experimental and may not even work properly in some case.
# Use at your own discretion.

	#Define what a CHUNK is, duh
	set EDMONDSON "Reflections CHUNK"

# CHUNK loading
	
	getdstring CHNKNM 0x04
	
#If it's not a Reflections CHUNK, don't load it!	
	if CHNKNM = CHNK
		set TYPE 0
	else
		print "\nERROR: Not a %EDMONDSON% File!\nHeader is %CHNKNM%, please correct this."
		cleanexit
	endif
	
#If it's a Reflections CHUNK, go ahead and load it!
	if TYPE = 0
			get CHNKSZ long
			get NUMPAKS long
			get DUMMY long
			print "\nType: 	%EDMONDSON%\nSize: 	%CHNKSZ%kb\n# Packages: 	%NUMPAKS%\n"
	endif

# Load the packages

goto 0x10
		
for i = 0 < NUMPAKS

	getdstring PAKNAME 0x04
	get PAKOFFSET long
	get PAKTYPE short
	get DUMMY short
	get PAKSIZE long
	
	# Files need a unique identifier and extension

	set DOMATH long
	set SUFFIX ".CHUNK" #Assign .CHUNK as file extension
	
	math DOMATH += PAKOFFSET #get offset
	string PAKNAME += DOMATH #assign offset to filename
	string PAKNAME += SUFFIX #assign .CHUNK to end of file ;)
	
	log PAKNAME PAKOFFSET PAKSIZE
	
next i

It lists all of the primary "packages" inside of the CHUNK. The new one will be able to dig deeper down into the files, and be able to expose parents and children - still working on my research document(s) so I'll hold off on that.

Posted: **Thu Nov 22, 2012 4:46 am**

if you're just posting a work-in-progress of your own i will leave you to it

if you need a hand send me a pm or something

Posted: **Thu Nov 22, 2012 5:16 am**

No no WRS, I'm actually quite excited you're looking into this stuff. I'm working on research and practicing BMS because I want to take up programming and stuff, so I gotta start somewhere. You're more than welcome to post research here, it'll help get the format cracked faster!

I didn't mean "quite unexpected" in a bad way now that I read it again. I just meant that it came to me as a shock that people were actually willing to help out, which is pretty neat.

Maybe you could take a stab at the more complex files? The most I can do is decode how everything is stored, cracking the formats is nearly impossible for someone like me. For example, open one of the .VVS files inside the "Vehicles" folder and look for "AWHF" - this is all the data for an individual vehicle, stored in some type of .X file format that's been modified to work differently.

Posted: **Sat Nov 24, 2012 2:30 am**

i started writing a small program to explore the chunks a little better - https://github.com/x1nixmzeng/Explor3r

it essentially does what you documented here: viewtopic.php?p=81162#p81162 using recursion to identify which CHNK blocks are themselves CHNK blocks

Posted: **Sat Nov 24, 2012 8:15 am**

This program is pretty much exactly what I had in mind! Excellent work!

Just a quick note, that unknown long that's at the end of the "CHNK" line isn't the version number - it's the same in Parallel Lines and San Francisco. I wonder what it's meant for.

Posted: **Tue Nov 27, 2012 1:43 am**

minor update

slow progress with the model data

the renderer model package chunks (MDPC) contain the models (faces indices, vertex data and textures) but i'm stuck on reconstructing them. all i think i've found is the normal data:

Posted: **Tue Nov 27, 2012 3:36 am**

That's definitely impressive. I think it may help to know that the model format is very similar to an .X file.

Posted: **Tue Nov 27, 2012 8:52 am**

Awesome! It will be simply awesome if there would be an import function aswell. Explor3r works perfect, kudos WRS

PS
Just one thing - can you rather make it extract to .DDS or .TGA format which is more comfortable for us modders?

Posted: **Tue Nov 27, 2012 6:31 pm**

@WRS is that from Driv3r or DriverSF?

Posted: **Tue Nov 27, 2012 7:21 pm**

Chipicao it's pretty obvious it's from DRIV3R, since he has no access to a copy of Driver: San Francisco (as far as I'm aware of).

XeNTaX

Reflections CHUNK File Format

Reflections CHUNK File Format

Re: Reflections CHUNK File Format

Re: Reflections CHUNK File Format

Re: Reflections CHUNK File Format

Re: Reflections CHUNK File Format

Re: Reflections CHUNK File Format

Re: Reflections CHUNK File Format

Re: Reflections CHUNK File Format

Re: Reflections CHUNK File Format

Re: Reflections CHUNK File Format

Re: Reflections CHUNK File Format

Re: Reflections CHUNK File Format

Re: Reflections CHUNK File Format

Re: Reflections CHUNK File Format

Re: Reflections CHUNK File Format