Beyond: Two Souls - Reversing | Porting (THAT'S ALL FOLKS?)

Programs that are related to this or that game.
User avatar
HugoPeters
veteran
Posts: 132
Joined: Thu Apr 03, 2014 9:49 am
Location: The Netherlands
Has thanked: 64 times
Been thanked: 168 times
Contact:

Re: [PS3] Beyond: Two Souls - Extractor | Reversing | Portin

Post by HugoPeters » Wed May 28, 2014 7:20 pm

Hamers wrote:Well that would be the best thing ever probably. But how? I mean the renderer is obviously pretty advanced and custom tailored to the SPUs right? How would you make that running on PC? Or do you plan to write an emulator specifically for Beyond :D
I'm not going to use their renderer, I'm only using the main executable as reference (the disassembled output is very useful).
If I can remake the way the game parses content I can make a new renderer using Mogre.
Image

"I'm going to throw my shit at you" —The Great Mighty Poo
_______________________>> check my portfolio I wish the [center] bbcode was turned on
_______________________>> or my twitter

User avatar
HugoPeters
veteran
Posts: 132
Joined: Thu Apr 03, 2014 9:49 am
Location: The Netherlands
Has thanked: 64 times
Been thanked: 168 times
Contact:

Re: [PS3] Beyond: Two Souls - Extractor | Reversing | Portin

Post by HugoPeters » Wed May 28, 2014 9:39 pm

Update! Been looking into Lua, and been porting / understanding Beyond's kernel code, and I can almost safely say the communicators are just Lua metatables of the inidivudal communicator tables. These tables have their own functions, variables, etc. Those functions look like they're just wrappers of C++ engine/kernel level methods.

Aaaanyhooow, here's some ported shit:

Image

In the region "game manager test" is around the first 400 lines of the actual Game Manager, using that as temp until I build the parser.

Here's the debug output...

Image

Not that much to see, but everything works just as expected and seems to work exactly like the game's (decompiled) source does.
Image

"I'm going to throw my shit at you" —The Great Mighty Poo
_______________________>> check my portfolio I wish the [center] bbcode was turned on
_______________________>> or my twitter

Hamers
n00b
Posts: 19
Joined: Fri Jul 22, 2011 4:39 pm
Has thanked: 21 times
Been thanked: 13 times

Re: [PS3] Beyond: Two Souls - Extractor | Reversing | Portin

Post by Hamers » Wed May 28, 2014 11:33 pm

HugoPeters wrote:
Hamers wrote:Well that would be the best thing ever probably. But how? I mean the renderer is obviously pretty advanced and custom tailored to the SPUs right? How would you make that running on PC? Or do you plan to write an emulator specifically for Beyond :D
I'm not going to use their renderer, I'm only using the main executable as reference (the disassembled output is very useful).
If I can remake the way the game parses content I can make a new renderer using Mogre.
Wow :eek: Well, if you can really do this, that probably would be indeed the best thing ever. Good luck :D
Image

User avatar
HugoPeters
veteran
Posts: 132
Joined: Thu Apr 03, 2014 9:49 am
Location: The Netherlands
Has thanked: 64 times
Been thanked: 168 times
Contact:

Re: [PS3] Beyond: Two Souls - Extractor | Reversing | Portin

Post by HugoPeters » Sat Jun 07, 2014 12:33 pm

Coming Soon... a brand new tool for viewing Beyond's files. Much more user friendly and intuitive.

I decided to do a complete rewrite as I know so much more now that I can make a much, much better and more stable core, a richer feature set and of course that more intuitive and user friendly layout.

To give a small example of the improved core: the total amount of lines used for reading the IDM files (encrypted to SDAT) used to be 273 in InfraTool.
In this new tool, there are just 50 lines spend on that!
That's a reducement of 82%!

Here's a screenshot from the debugger of a tiny part of the parser that doesn't really say anything at all but does kinda give a little clue of how I want to "order" the files in this new tool.

Image


And yeah, of course I made a logo.

Image
Image

"I'm going to throw my shit at you" —The Great Mighty Poo
_______________________>> check my portfolio I wish the [center] bbcode was turned on
_______________________>> or my twitter

User avatar
Mr.Mouse
Site Admin
Posts: 4036
Joined: Wed Jan 15, 2003 6:45 pm
Location: Dungeons of Doom
Has thanked: 413 times
Been thanked: 548 times
Contact:

Re: [PS3] Beyond: Two Souls - Extractor | Reversing | Portin

Post by Mr.Mouse » Mon Jun 09, 2014 10:57 am

Looking good. 8D

User avatar
HugoPeters
veteran
Posts: 132
Joined: Thu Apr 03, 2014 9:49 am
Location: The Netherlands
Has thanked: 64 times
Been thanked: 168 times
Contact:

Re: [PS3] Beyond: Two Souls - Extractor | Reversing | Portin

Post by HugoPeters » Sun Jun 22, 2014 11:55 pm

PLOT TWIST #1.

=====> http://youtu.be/aVO_eiPiRuY

I have (partially) reversed the sequence / movie timeline format, demo at around 2/5 of the video.

And also: new tool is pretty neat.
Image

"I'm going to throw my shit at you" —The Great Mighty Poo
_______________________>> check my portfolio I wish the [center] bbcode was turned on
_______________________>> or my twitter

JayK
veteran
Posts: 122
Joined: Fri Jun 01, 2012 10:08 am
Has thanked: 31 times
Been thanked: 65 times

Re: [PS3] Beyond: Two Souls - Extractor | Reversing | Portin

Post by JayK » Mon Jun 23, 2014 7:34 am

Glad to see an update from you, the progress you've made on your tool is really amazing, it looks very well constructed. Keep it up and good luck.

User avatar
HugoPeters
veteran
Posts: 132
Joined: Thu Apr 03, 2014 9:49 am
Location: The Netherlands
Has thanked: 64 times
Been thanked: 168 times
Contact:

Re: [PS3] Beyond: Two Souls - Extractor | Reversing | Portin

Post by HugoPeters » Mon Jun 23, 2014 9:59 am

JayK wrote:Glad to see an update from you, the progress you've made on your tool is really amazing, it looks very well constructed. Keep it up and good luck.
Thanks. The "movies" in Beyond are basically cutscenes like the Prologue, but are also the things that contain choices, button-press-events, etc. As you can see in the video, I can get the the Prologue's audio to play together with all dialog. It's all separate pieces of dialog, with time indices into the dialog audio. I also found the indices of the subtitles (which is an array of start - end floats) and the localization text gives indices into these floats, so not everything is used.
I can also already parse the camera shots, so I know what camera shot starts when and what properties it has (thanks QD for leaving the property names in there :twisted:).
If we can get the textures out, it should be just a matter of time before we can get whole sequences and parts of the game to play.

If anyone knows anything about JPEG / DXT / DDS / compression, please check out: viewtopic.php?f=21&t=11607
Image

"I'm going to throw my shit at you" —The Great Mighty Poo
_______________________>> check my portfolio I wish the [center] bbcode was turned on
_______________________>> or my twitter

User avatar
HugoPeters
veteran
Posts: 132
Joined: Thu Apr 03, 2014 9:49 am
Location: The Netherlands
Has thanked: 64 times
Been thanked: 168 times
Contact:

Re: [PS3] Beyond: Two Souls - Extractor | Reversing | Portin

Post by HugoPeters » Wed Jun 25, 2014 5:49 pm

-
-
-

Just in before the weekly update, this should clear things up (or not)
-
-
-


edit: you may have noticed the audio is a bit off and the dialog is a bit choppy, this has been corrected now.
Image

"I'm going to throw my shit at you" —The Great Mighty Poo
_______________________>> check my portfolio I wish the [center] bbcode was turned on
_______________________>> or my twitter

User avatar
HugoPeters
veteran
Posts: 132
Joined: Thu Apr 03, 2014 9:49 am
Location: The Netherlands
Has thanked: 64 times
Been thanked: 168 times
Contact:

Re: [PS3] Beyond: Two Souls - Extractor | Reversing | Portin

Post by HugoPeters » Fri Jun 27, 2014 9:57 am

Spoilers!!!
See here, what's possible right now: https://www.youtube.com/watch?v=4ikhbv4xkGk
Dialog is still off here and there, will try to fix today. Also the loop points are incorrect so is the volume of some parts.
Image

"I'm going to throw my shit at you" —The Great Mighty Poo
_______________________>> check my portfolio I wish the [center] bbcode was turned on
_______________________>> or my twitter

User avatar
HugoPeters
veteran
Posts: 132
Joined: Thu Apr 03, 2014 9:49 am
Location: The Netherlands
Has thanked: 64 times
Been thanked: 168 times
Contact:

Re: [PS3] Beyond: Two Souls - Extractor | Reversing | Portin

Post by HugoPeters » Tue Jul 01, 2014 12:04 pm

Here's a nice extended demo video of the parsed timeline: https://www.youtube.com/watch?v=tW9b_UE9Pd4

chrrox and I are working on models as we speak, and I can confirm it's exactly as how I would expect them to do it.

<catalog>
skeleton
_____particles
_____nodes
_____mesh table
__________<link to compressed mesh files>
__________meshdata
_______________shaders
____________________textures
_______________edge data

So catalogs only have one skeleton which links to meshes, etc. All models have 1 skeleton only so it serves as the root entity (thanks debug strings :D). Area's have more then one skeleton (for all static objects) so the dev would have to specify the root entity (again, thanks debug strings!)

Soon more? :D But no textures unless we manage to figure out the texture format/compression. Anyone who knows PPC/elf assembly? :scaredy:
Last edited by HugoPeters on Wed Jul 02, 2014 7:02 pm, edited 1 time in total.
Image

"I'm going to throw my shit at you" —The Great Mighty Poo
_______________________>> check my portfolio I wish the [center] bbcode was turned on
_______________________>> or my twitter

User avatar
HugoPeters
veteran
Posts: 132
Joined: Thu Apr 03, 2014 9:49 am
Location: The Netherlands
Has thanked: 64 times
Been thanked: 168 times
Contact:

Re: [PS3] Beyond: Two Souls - Extractor | Reversing | Portin

Post by HugoPeters » Wed Jul 02, 2014 7:00 pm

PLOT TWIST #2

Surprise.

Image

Image
Image

"I'm going to throw my shit at you" —The Great Mighty Poo
_______________________>> check my portfolio I wish the [center] bbcode was turned on
_______________________>> or my twitter

User avatar
RunaWhite
veteran
Posts: 156
Joined: Sat Jan 07, 2012 2:30 pm
Has thanked: 55 times
Been thanked: 31 times

Re: Beyond: Two Souls - Reversing | Porting (UPDATE: Models!

Post by RunaWhite » Thu Jul 03, 2014 12:32 am

OMG so are the models possible?? This is awesome. I hope it'll be possible to get them also with bones and weights... and that it's going to be the same format as Heavy Rain (so I read). I've been waiting for those models for ages.

Thank you very much for all the hard work you guys do on this stuff!

User avatar
o0Crofty0o
veteran
Posts: 159
Joined: Wed Nov 17, 2010 2:57 pm
Location: Germany
Has thanked: 135 times
Been thanked: 50 times

Re: Beyond: Two Souls - Reversing | Porting (UPDATE: Models!

Post by o0Crofty0o » Fri Jul 04, 2014 10:51 am

Wow thanks so much for all the work on this :o This is amazing ;_;

User avatar
HugoPeters
veteran
Posts: 132
Joined: Thu Apr 03, 2014 9:49 am
Location: The Netherlands
Has thanked: 64 times
Been thanked: 168 times
Contact:

Re: Beyond: Two Souls - Reversing | Porting (UPDATE: Models!

Post by HugoPeters » Sun Jul 06, 2014 12:26 pm

Models Update!

Big thanks to chrrox and howfie for helping me getting this working!!

I ported chrrox's PSA edge index decompression over to C#, and when I finally got my hand on a working bitreader and getting the normals correct (really big thanks to chrrox and howfie for that!) I succeeded in getting the models to show up!

I will be using Mogre as my graphics rendering engine (and it's fucking awesome).

Image Image
Image
(the arm missing is because there's cloth physics there)

Image



And here's a glimpse into the future...

Image
Image

"I'm going to throw my shit at you" —The Great Mighty Poo
_______________________>> check my portfolio I wish the [center] bbcode was turned on
_______________________>> or my twitter

Post Reply