READ THE RULES: Click here

Follow us on Facebook: https://www.facebook.com/xentax/ :)

Heavy Rain Tool

Programs that are related to this or that game.
MushroomLT
ultra-n00b
Posts: 5
Joined: Sat Sep 02, 2017 1:09 pm
Has thanked: 2 times
Been thanked: 10 times

Heavy Rain Tool

Post by MushroomLT » Sun Sep 03, 2017 2:42 pm

The beginnings of the ultimate Heavy Rain extractor tool, or even PC port in the future - Heavy Rain Tool

Image

After finding @HugoPeters work on Beyond, I became really interested in the concept of reverse-engineering a game engine, so I started working on my favourite game - Heavy Rain.

Currently I managed to:
  • Read the BigFile index (the sdat encrypted file)
  • Find and display a list of entries grouped by type
  • Unpack all of the segs archived entries
  • Read the inner structure of the entries
  • Parse some of the blocks of the entries to extract data from them:
  • Textures (some of them are misaligned)
  • Scripts that are not compiled
  • BIK Video files
  • Some of the audio files + Multi-Channel audio exporting (Mostly soundtrack)
What I'm working on:
  • Linking the entries together by parsing the storyboard
  • Properly aligning textures
(Last updated: 2017-09-12, PreAlpha 3)

Tool usage:
* Run the tool to generate "engine.json" configuration file.
* Open the configuration file and set "qdGameDirectory" to the game file directory location
* Set vlcExeLocation to the location of your VLC Media Player installation to be able to play sounds directly from the tool
* Set loadBigFile to the BigFile index file you want to load.

The tool can extract any entry into a separate binary file by selecting an entry and clicking the Export button.
To view the inner blocks of the entry double click it in the tree. Selecting a block will automatically adjust the offset in the hex view.
To export video from an entry double click the entry, and double click the VIDEDATA block.
To export sounds from an entry double click the entry, and double click the PARTITIO block.
To play the exported sound using VLC player, double click the entry inside of the PARTITIO block.

To view scripts, double click the SCRIPT block, and select the inner script entry, it will be displayed under the Contents tab.
Last edited by MushroomLT on Wed Mar 20, 2019 8:17 pm, edited 4 times in total.

MushroomLT
ultra-n00b
Posts: 5
Joined: Sat Sep 02, 2017 1:09 pm
Has thanked: 2 times
Been thanked: 10 times

Some progress

Post by MushroomLT » Fri Sep 08, 2017 5:17 pm

I managed to perform a RAM dump of RPCS3 running the game by modifying it's source, and from the dump I found the LUA initialisation code for probably all of the data types. Now the tool is capable of displaying what data each entry has:

Code: Select all

CommunicatorBlock Contents:
DCINFO Entries: 
	Entry 0: SOUND_OFFSET; 64112
	Entry 1: SOUND_DATA; 64108
	Entry 2: SCRIPT_SOUND_ENTITY; 15116
	Entry 3: SOUND_OFFSET; 64116
	Entry 4: SOUND_DATA; 64112
	Entry 5: SCRIPT_SOUND_ENTITY; 15118
	Entry 6: SOUND_OFFSET; 66806
	Entry 7: SOUND_DATA; 66803
	Entry 8: SCRIPT_SOUND_ENTITY; 15117
	Entry 9: SOUND_OFFSET; 46980
	Entry 10: SOUND_DATA; 46976
	Entry 11: SCRIPT_SOUND_ENTITY; 10944
	Entry 12: SOUND_OFFSET; 64114
	Entry 13: SOUND_DATA; 64110
	Entry 14: SCRIPT_SOUND_ENTITY; 15120
	Entry 15: SOUND_OFFSET; 62123
	Entry 16: SOUND_DATA; 62119
	Entry 17: SCRIPT_SOUND_ENTITY; 10947
	Entry 18: SCRIPT_SOUND_ENTITY; 10945
	Entry 19: SCRIPT_SOUND_ENTITY; 15119
	Entry 20: LOADING_ZONE; 5681
	Entry 21: LOADING_ZONE; 5756
	Entry 22: LOADING_ZONE; 6602
	Entry 23: LOADING_ZONE; 6630
	Entry 24: LOADING_ZONE; 6676
	Entry 25: LOADING_ZONE; 6718
	Entry 26: LOADING_ZONE; 6767
	Entry 27: LOADING_ZONE; 6779
	Entry 28: LOADING_ZONE; 22721
	Entry 29: LOADING_ZONE; 22724
	Entry 30: LOADING_ZONE; 22727
	Entry 31: LOADING_ZONE; 22729
	Entry 32: LOADING_ZONE; 22754
	Entry 33: LOADING_ZONE; 38606
	Entry 34: STORYBOARD; 46
	Entry 35: LOADING_ZONE; 5674
	Entry 36: LOADING_ZONE; 5675
	Entry 37: LOADING_ZONE; 5677
	Entry 38: LOADING_ZONE; 5678
	Entry 39: LOADING_ZONE; 5679
	Entry 40: LOADING_ZONE; 5689
	Entry 41: LOADING_ZONE; 5690
	Entry 42: LOADING_ZONE; 5691
	Entry 43: LOADING_ZONE; 5704
	Entry 44: LOADING_ZONE; 5739
	Entry 45: LOADING_ZONE; 5740
	Entry 46: LOADING_ZONE; 5741
	Entry 47: LOADING_ZONE; 5754
	Entry 48: LOADING_ZONE; 6498
	Entry 49: LOADING_ZONE; 6569
	Entry 50: LOADING_ZONE; 6659
	Entry 51: LOADING_ZONE; 6662
	Entry 52: LOADING_ZONE; 6669
	Entry 53: LOADING_ZONE; 6670
	Entry 54: LOADING_ZONE; 6679
	Entry 55: LOADING_ZONE; 6686
	Entry 56: LOADING_ZONE; 6689
	Entry 57: LOADING_ZONE; 6696
	Entry 58: LOADING_ZONE; 6697
	Entry 59: LOADING_ZONE; 6698
	Entry 60: LOADING_ZONE; 6701
	Entry 61: LOADING_ZONE; 6703
	Entry 62: LOADING_ZONE; 6705
	Entry 63: LOADING_ZONE; 6706
	Entry 64: LOADING_ZONE; 6707
	Entry 65: LOADING_ZONE; 6709
	Entry 66: LOADING_ZONE; 6711
	Entry 67: LOADING_ZONE; 6714
	Entry 68: LOADING_ZONE; 6716
	Entry 69: LOADING_ZONE; 6719
	Entry 70: LOADING_ZONE; 6720
	Entry 71: LOADING_ZONE; 6723
	Entry 72: LOADING_ZONE; 6724
	Entry 73: LOADING_ZONE; 6725
	Entry 74: LOADING_ZONE; 6728
	Entry 75: LOADING_ZONE; 6730
	Entry 76: LOADING_ZONE; 6732
	Entry 77: LOADING_ZONE; 6733
	Entry 78: LOADING_ZONE; 6734
	Entry 79: LOADING_ZONE; 6738
	Entry 80: LOADING_ZONE; 6740
	Entry 81: LOADING_ZONE; 6742
	Entry 82: LOADING_ZONE; 6745
	Entry 83: LOADING_ZONE; 6748
	Entry 84: LOADING_ZONE; 6753
	Entry 85: LOADING_ZONE; 6755
	Entry 86: LOADING_ZONE; 6758
	Entry 87: LOADING_ZONE; 6760
	Entry 88: LOADING_ZONE; 6771
	Entry 89: LOADING_ZONE; 6773
	Entry 90: LOADING_ZONE; 6774
	Entry 91: LOADING_ZONE; 6776
	Entry 92: LOADING_ZONE; 6777
	Entry 93: LOADING_ZONE; 6785
	Entry 94: LOADING_ZONE; 6788
	Entry 95: LOADING_ZONE; 6789
	Entry 96: LOADING_ZONE; 6792
	Entry 97: LOADING_ZONE; 6795
	Entry 98: LOADING_ZONE; 6797
	Entry 99: LOADING_ZONE; 6798
	Entry 100: LOADING_ZONE; 6799
	Entry 101: LOADING_ZONE; 6800
	Entry 102: LOADING_ZONE; 6801
	Entry 103: LOADING_ZONE; 6805
	Entry 104: LOADING_ZONE; 6807
	Entry 105: LOADING_ZONE; 6808
	Entry 106: LOADING_ZONE; 6810
	Entry 107: LOADING_ZONE; 6811
	Entry 108: LOADING_ZONE; 6814
	Entry 109: LOADING_ZONE; 6816
	Entry 110: LOADING_ZONE; 6817
	Entry 111: LOADING_ZONE; 6818
	Entry 112: LOADING_ZONE; 6825
	Entry 113: LOADING_ZONE; 6828
	Entry 114: LOADING_ZONE; 6830
	Entry 115: LOADING_ZONE; 6832
	Entry 116: LOADING_ZONE; 6834
	Entry 117: LOADING_ZONE; 6835
	Entry 118: LOADING_ZONE; 6837
	Entry 119: LOADING_ZONE; 7700
	Entry 120: LOADING_ZONE; 10309
	Entry 121: LOADING_ZONE; 10419
	Entry 122: LOADING_ZONE; 17933
	Entry 123: LOADING_ZONE; 18595
	Entry 124: LOADING_ZONE; 20832
	Entry 125: LOADING_ZONE; 20834
	Entry 126: LOADING_ZONE; 20836
	Entry 127: LOADING_ZONE; 20850
	Entry 128: LOADING_ZONE; 20852
	Entry 129: LOADING_ZONE; 22359
	Entry 130: LOADING_ZONE; 23752
	Entry 131: LOADING_ZONE; 23756
	Entry 132: LOADING_ZONE; 23761
	Entry 133: LOADING_ZONE; 23763
	Entry 134: LOADING_ZONE; 23969
	Entry 135: LOADING_ZONE; 25561
	Entry 136: LOADING_ZONE; 28431
	Entry 137: LOADING_ZONE; 29901
	Entry 138: LOADING_ZONE; 29907
	Entry 139: LOADING_ZONE; 29909
	Entry 140: LOADING_ZONE; 29911
	Entry 141: LOADING_ZONE; 29913
	Entry 142: LOADING_ZONE; 29915
	Entry 143: LOADING_ZONE; 29917
	Entry 144: LOADING_ZONE; 29920
	Entry 145: LOADING_ZONE; 29924
	Entry 146: LOADING_ZONE; 29928
	Entry 147: LOADING_ZONE; 29932
	Entry 148: LOADING_ZONE; 29937
	Entry 149: LOADING_ZONE; 29953
	Entry 150: LOADING_ZONE; 29956
	Entry 151: LOADING_ZONE; 33069
	Entry 152: LOADING_ZONE; 33215
	Entry 153: LOADING_ZONE; 33239
	Entry 154: LOADING_ZONE; 33241
	Entry 155: LOADING_ZONE; 34655
	Entry 156: LOADING_ZONE; 35557
	Entry 157: LOADING_ZONE; 35863
	Entry 158: LOADING_ZONE; 38507
	Entry 159: LOADING_ZONE; 39037
	Entry 160: LOADING_ZONE; 39041
	Entry 161: LOADING_ZONE; 39269
	Entry 162: LOADING_ZONE; 39448
	Entry 163: LOADING_ZONE; 39519
	Entry 164: LOADING_ZONE; 39521
	Entry 165: LOADING_ZONE; 39523
	Entry 166: LOADING_ZONE; 40373
	Entry 167: STORYBOARD; 35
	Entry 168: ENGINE_TEXTURE_FILE; 49948
	Entry 169: ENGINE_TEXTURE_FILE; 50553
	Entry 170: ENGINE_TEXTURE_FILE; 54206
	Entry 171: ENGINE_TEXTURE_FILE; 54207
	Entry 172: ENGINE_TEXTURE_FILE; 54208
	Entry 173: ENGINE_TEXTURE_FILE; 54209
	Entry 174: ENGINE_TEXTURE_FILE; 54210
	Entry 175: ENGINE_TEXTURE_FILE; 54211
	Entry 176: ENGINE_TEXTURE_FILE; 54212
	Entry 177: ENGINE_TEXTURE_FILE; 54213
	Entry 178: ENGINE_TEXTURE_FILE; 54214
	Entry 179: ENGINE_TEXTURE_FILE; 62181
	Entry 180: ENGINE_TEXTURE_FILE; 62182
	Entry 181: ENGINE_TEXTURE_FILE; 62183
	Entry 182: ENGINE_TEXTURE_FILE; 62184
	Entry 183: ENGINE_TEXTURE_FILE; 62185
	Entry 184: ENGINE_TEXTURE_FILE; 62186
	Entry 185: ENGINE_TEXTURE_FILE; 62187
	Entry 186: ENGINE_TEXTURE_FILE; 62188
	Entry 187: ENGINE_TEXTURE_FILE; 62921
	Entry 188: ENGINE_TEXTURE_FILE; 64307
	Entry 189: ENGINE_TEXTURE_FILE; 64308
	Entry 190: ENGINE_TEXTURE_FILE; 64309
	Entry 191: ENGINE_TEXTURE_FILE; 69866
	Entry 192: ENGINE_TEXTURE_FILE; 75541
	Entry 193: ENGINE_TEXTURE_FILE; 75542
	Entry 194: ENGINE_TEXTURE_FILE; 75771
	Entry 195: ENGINE_TEXTURE_FILE; 76140
	Entry 196: ENGINE_TEXTURE_FILE; 78367
	Entry 197: ENGINE_TEXTURE_FILE; 78368
	Entry 198: ENGINE_TEXTURE_FILE; 78369
	Entry 199: ENGINE_TEXTURE_FILE; 78370
	Entry 200: ENGINE_TEXTURE_FILE; 78371
	Entry 201: ENGINE_TEXTURE_FILE; 78372
	Entry 202: ENGINE_TEXTURE_FILE; 78373
	Entry 203: ENGINE_TEXTURE_FILE; 78374
	Entry 204: ENGINE_TEXTURE_FILE; 78375
	Entry 205: ENGINE_TEXTURE_FILE; 78376
	Entry 206: ENGINE_TEXTURE_FILE; 78377
	Entry 207: ENGINE_TEXTURE_FILE; 78378
	Entry 208: ENGINE_TEXTURE_FILE; 79345
	Entry 209: ENGINE_TEXTURE_FILE; 62189
	Entry 210: ENGINE_TEXTURE_FILE; 69848
	Entry 211: MENUS_MASTER_FONT_DATA; 9
	Entry 212: MENUS_MASTER_DATA; 28
	Entry 213: CHARACTER; 3
	Entry 214: CHARACTER; 345
	Entry 215: CHARACTER; 7
	Entry 216: CHARACTER; 6

DCDATA Size: 5336713; Entries: 7
	Entry 0: 0
	Entry 1: 48
	Entry 2: 0
	Entry 3: 0
	Entry 4: 256
	Entry 5: 0
	Entry 6: 1

MushroomLT
ultra-n00b
Posts: 5
Joined: Sat Sep 02, 2017 1:09 pm
Has thanked: 2 times
Been thanked: 10 times

Textures reversed

Post by MushroomLT » Fri Sep 08, 2017 9:28 pm

Textures reversed!
Image

WRS
ultra-veteran
ultra-veteran
Posts: 601
Joined: Fri Nov 06, 2009 12:13 am
Has thanked: 74 times
Been thanked: 132 times

Re: Heavy Rain Tool

Post by WRS » Sat Sep 09, 2017 10:18 pm

Awesome keep it up!
Useful tool links:

MushroomLT
ultra-n00b
Posts: 5
Joined: Sat Sep 02, 2017 1:09 pm
Has thanked: 2 times
Been thanked: 10 times

Re: Heavy Rain Tool

Post by MushroomLT » Tue Sep 12, 2017 1:54 pm

DataContainer structure reversed - working on the storyboard now, which links all the scenes.
Image

User avatar
HugoPeters
veteran
Posts: 133
Joined: Thu Apr 03, 2014 9:49 am
Location: The Netherlands
Has thanked: 64 times
Been thanked: 168 times
Contact:

Re: Heavy Rain Tool

Post by HugoPeters » Sat Sep 16, 2017 11:25 pm

Great to see my stuff inspired a follow-up! :up:
Image

"I'm going to throw my shit at you" —The Great Mighty Poo
_______________________>> check my portfolio I wish the [center] bbcode was turned on
_______________________>> or my twitter

Hamers
beginner
Posts: 21
Joined: Fri Jul 22, 2011 4:39 pm
Has thanked: 21 times
Been thanked: 13 times

Re: Heavy Rain Tool

Post by Hamers » Sun Sep 17, 2017 11:55 am

Awesome work dude! Also these are the scenes that were cut from the game in Beta status, fully playable but unpolished:

0501 Sleepless Night Underwater
0701 Going to School Wakeup
1002 Swimming Again

Since they have almost been completed and use mostly existing assets, everything you could find in regards to them would be quite amazing. I doubt you can fully restore them, but maybe they have a lot of traces left :D
Image

michalss
Moderator
Posts: 941
Joined: Sun Mar 27, 2011 8:42 pm
Has thanked: 10 times
Been thanked: 134 times

Re: Heavy Rain Tool

Post by michalss » Sun Sep 17, 2017 8:42 pm

well i hope u finnish it since Hugo had to drop it.. :)
Quick BMS Editor GUI - simple easy to use
Goto : viewtopic.php?uid=34229&f=29&t=6797&start=0

Downloads from DropBox : https://dl.dropboxusercontent.com/u/

oyunceviri
advanced
Posts: 67
Joined: Tue Jun 30, 2009 11:35 am
Has thanked: 28 times
Been thanked: 5 times

Re: Heavy Rain Tool

Post by oyunceviri » Sun Sep 24, 2017 11:38 pm

Any news?

daemon1
M-M-M-Monster veteran
M-M-M-Monster veteran
Posts: 1898
Joined: Tue Mar 24, 2015 8:12 pm
Has thanked: 48 times
Been thanked: 1457 times

Re: Heavy Rain Tool

Post by daemon1 » Sun Oct 01, 2017 1:23 pm

Quantic Dream game formats were reversed all the way, and tools released to public. But it was forbidden to talk about this engine here on xentax. Any changes?

Image
Image

User avatar
HugoPeters
veteran
Posts: 133
Joined: Thu Apr 03, 2014 9:49 am
Location: The Netherlands
Has thanked: 64 times
Been thanked: 168 times
Contact:

Re: Heavy Rain Tool

Post by HugoPeters » Mon Oct 02, 2017 4:06 pm

daemon1 wrote:Quantic Dream game formats were reversed all the way, and tools released to public. But it was forbidden to talk about this engine here on xentax. Any changes?

<snip>
No they were not, there was only a model extractor which just brute-force scans files for model data.
Image

"I'm going to throw my shit at you" —The Great Mighty Poo
_______________________>> check my portfolio I wish the [center] bbcode was turned on
_______________________>> or my twitter

daemon1
M-M-M-Monster veteran
M-M-M-Monster veteran
Posts: 1898
Joined: Tue Mar 24, 2015 8:12 pm
Has thanked: 48 times
Been thanked: 1457 times

Re: Heavy Rain Tool

Post by daemon1 » Mon Oct 02, 2017 4:18 pm

HugoPeters wrote:
daemon1 wrote:Quantic Dream game formats were reversed all the way, and tools released to public. But it was forbidden to talk about this engine here on xentax. Any changes?

<snip>
No they were not, there was only a model extractor which just brute-force scans files for model data.
Yes, they were. You probably just don't know about that. And a proper tool was released which is able to extract all models and levels with textures, skeletons and fully weighted including cloth, from both Heavy Rain & Beyond Two Souls. I posted images with models extracted with that tool.

My question is: can I now talk about this on xentax?

Modman69
veteran
Posts: 104
Joined: Wed Jun 17, 2009 4:33 pm
Has thanked: 18 times
Been thanked: 3 times

Re: Heavy Rain Tool

Post by Modman69 » Tue Oct 03, 2017 6:31 am

I think the tool was on the Cg in Games forum (QDtools)

but it seems it cannot be downloaded any longer

It's to bad since I sold my copy of Heavy Rain for PS3 a while back because I figured there was never going to be a way to get the models
I would probably go buy another copy if I knew the tools were available
(actually I bet a lot of people might buy a copy)

daemon1
M-M-M-Monster veteran
M-M-M-Monster veteran
Posts: 1898
Joined: Tue Mar 24, 2015 8:12 pm
Has thanked: 48 times
Been thanked: 1457 times

Re: Heavy Rain Tool

Post by daemon1 » Tue Oct 03, 2017 4:16 pm

Modman69 wrote:I think the tool was on the Cg in Games forum (QDtools)
Tools can still be downloaded from CG and also Facepunch, and maybe other sites.

User avatar
HugoPeters
veteran
Posts: 133
Joined: Thu Apr 03, 2014 9:49 am
Location: The Netherlands
Has thanked: 64 times
Been thanked: 168 times
Contact:

Re: Heavy Rain Tool

Post by HugoPeters » Wed Oct 04, 2017 10:50 am

daemon1 wrote:
HugoPeters wrote:
daemon1 wrote:Quantic Dream game formats were reversed all the way, and tools released to public. But it was forbidden to talk about this engine here on xentax. Any changes?

<snip>
No they were not, there was only a model extractor which just brute-force scans files for model data.
Yes, they were. You probably just don't know about that. And a proper tool was released which is able to extract all models and levels with textures, skeletons and fully weighted including cloth, from both Heavy Rain & Beyond Two Souls. I posted images with models extracted with that tool.

My question is: can I now talk about this on xentax?
Again, no they were not. Apparently you think models are the only thing what makes a game ;)
The tool you're talking about, like I said, brute-force scans files for model data. I didn't say it doesn't produce proper models.
Give me a tool that can also interpret animation data and all the rest that comes with it and can produce a scene like I showcase on my portfolio (http://hugo.fyi/projects/reverse-engineering-b2s) - then we'll talk :bleh:
Image

"I'm going to throw my shit at you" —The Great Mighty Poo
_______________________>> check my portfolio I wish the [center] bbcode was turned on
_______________________>> or my twitter

Post Reply