The Last of Us Model Viewer/Extractor (PS4)

Post questions about game models here, or help out others!
Post Reply
akderebur
ultra-veteran
ultra-veteran
Posts: 324
Joined: Fri Jul 08, 2011 10:36 am
Has thanked: 44 times
Been thanked: 290 times

The Last of Us Model Viewer/Extractor (PS4)

Post by akderebur » Thu Feb 02, 2017 6:55 pm

UPDATE
- Added support for loading maps
- Fixed some problems with the characters
- Fixed "Remake UVs"

Hello everyone,

I was looking at The Last Of Us pak files, trying to find some sort of index table for vertex/face positions but I noticed that it is also possible to get this information by searching for patterns in the file, without actually using an index table.

So I made a program to find and view meshes from the pak files (ps4 version only). You can also export them as obj/iqe. I used Unity to be able to render the meshes (not very experienced with low level graphic libraries), that is why the program is a bit large.

First some screenshots to get attention :D

Ellie :
Image

David:
Image

UPDATED

How to Use?
- File -> Open to load the pak file
- In the first file select, select the pak file that has the model data (ex. "ellie-tstrm.pak")
- In the second file select, select the pak file that has the skeleton data (ex. "ellie-skel-tstrm.pak")
- Some models don't have separate skeleton files (like infected), instead the skeleton is inside the same file. In this case select the first pak file again.
- If you don't want the skeleton, or if you are loading a map, click "Cancel" for the second file select

The program will go through the file, finding meshes/skeleton and add them to the scene.

You can see the meshes that are currently in the scene, in a list box on the left. If you select a mesh from the list, it will get highlighted in the scene. Bones will also appear as white spheres but it is not possible to move/rotate the bones. They are just there to show you that the skeleton is loaded properly.

Normally the meshes should load fine. If for some reason one of the meshes appear distorted you can just disable it by checking the "Disable Mesh" box. Disabled meshes won't get exported.

Preview Controls
Right-click and move mouse : Orbit around model
Ctrl + Mouse Wheel Scroll : Zoom
Mouse Wheel Click and move mouse : Pan
W Key : Wireframe shader

Exporting
There are two export options:

Obj : This will export all meshes as separate obj files. Of course there will be no skinning data and skeleton.
IQE : This will export the model as a single iqe file. IQE isn't really popular but it is a format that is not complex and that supports skinning, so I went with it.
There is a blender importer script for iqe : https://raw.githubusercontent.com/ccxvi ... _import.py From blender you can pretty much export it to any format you want.

Download: http://www.mediafire.com/file/0b9i8nat4 ... r_Upd2.rar

Example model imported into Maya:
Image

I tried loading some files other than the characters, like props, weapons, animals etc. and they all seemed to work fine with minor problems. Still there might be issues with some of the files, because I didn't tested every one of them. If that is the case, let me know.

If you are also interested in getting the textures, check out my texture tool : viewtopic.php?f=18&t=17800

Credits
daemon1 - for figuring out all of the model data and explaining it really well with videos
luxox18 - for finding out data manually, so that I can use them as a reference
Last edited by akderebur on Sun May 13, 2018 2:33 pm, edited 6 times in total.

amzerof6
advanced
Posts: 57
Joined: Sat Oct 23, 2010 3:50 pm
Has thanked: 2 times
Been thanked: 2 times

Re: Last of Us Model Viewer/Extractor (PS4)

Post by amzerof6 » Mon Feb 13, 2017 3:18 pm

wow ,this is realy awsome!!

Jaydenthetank
veteran
Posts: 86
Joined: Tue Nov 06, 2012 7:57 am
Has thanked: 13 times
Been thanked: 6 times

Re: Last of Us Model Viewer/Extractor (PS4)

Post by Jaydenthetank » Thu Feb 16, 2017 2:05 am

Will this work on uncharted 4?

akderebur
ultra-veteran
ultra-veteran
Posts: 324
Joined: Fri Jul 08, 2011 10:36 am
Has thanked: 44 times
Been thanked: 290 times

Re: Last of Us Model Viewer/Extractor (PS4)

Post by akderebur » Thu Feb 16, 2017 2:19 am

Jaydenthetank wrote:Will this work on uncharted 4?
I don't know, it might work if the file structure is identical with the last of us. I don't have the uncharted files, so didn't have the chance to test them.

Jaydenthetank
veteran
Posts: 86
Joined: Tue Nov 06, 2012 7:57 am
Has thanked: 13 times
Been thanked: 6 times

Re: Last of Us Model Viewer/Extractor (PS4)

Post by Jaydenthetank » Thu Feb 16, 2017 6:53 am

Just curious because I don't think anyone has ripped ps4 models before

Darko
double-veteran
double-veteran
Posts: 701
Joined: Mon Jul 13, 2009 6:16 pm
Has thanked: 70 times
Been thanked: 118 times

Re: Last of Us Model Viewer/Extractor (PS4)

Post by Darko » Thu Feb 16, 2017 11:21 am

Jaydenthetank wrote:Just curious because I don't think anyone has ripped ps4 models before
https://www.youtube.com/watch?v=-Y1bigdq-6o&t
Image

User avatar
volfin
ultra-veteran
ultra-veteran
Posts: 417
Joined: Sun Jul 06, 2014 6:30 am
Has thanked: 100 times
Been thanked: 260 times

Re: Last of Us Model Viewer/Extractor (PS4)

Post by volfin » Sun Feb 19, 2017 12:10 am

akderebur wrote:
Jaydenthetank wrote:Will this work on uncharted 4?
I don't know, it might work if the file structure is identical with the last of us. I don't have the uncharted files, so didn't have the chance to test them.
Uncharted 4 is something I started looking into. But the whole game is just one single pak file, so I suspect it's not similar at all. EDIT: I found out it's because PS4 games are encrypted, and still nobody knows how to unencrypt them (which begs the question how you unencrypted this game?)

I am curious if you can release the source code or futher explain the findings, as I'm sure it would help with my own investigations, the more we can pool information, the farther we can go.

akderebur
ultra-veteran
ultra-veteran
Posts: 324
Joined: Fri Jul 08, 2011 10:36 am
Has thanked: 44 times
Been thanked: 290 times

Re: Last of Us Model Viewer/Extractor (PS4)

Post by akderebur » Sun Feb 19, 2017 2:43 am

volfin wrote:Uncharted 4 is something I started looking into. But the whole game is just one single pak file, so I suspect it's not similar at all. EDIT: I found out it's because PS4 games are encrypted, and still nobody knows how to unencrypt them (which begs the question how you unencrypted this game?)
I did not, luxox is the guy you are looking for :D

He released some character files in this thread : viewtopic.php?f=10&t=15801

User avatar
Wobble
ultra-veteran
ultra-veteran
Posts: 585
Joined: Tue Jan 04, 2005 9:47 pm
Has thanked: 43 times
Been thanked: 112 times

Re: Last of Us Model Viewer/Extractor (PS4)

Post by Wobble » Sun Feb 19, 2017 6:55 am

[out]
Last edited by Wobble on Sat Mar 11, 2017 11:17 am, edited 1 time in total.

User avatar
shakotay2
MEGAVETERAN
MEGAVETERAN
Posts: 2606
Joined: Fri Apr 20, 2012 9:24 am
Location: Nexus, searching for Jim Kirk
Has thanked: 638 times
Been thanked: 1332 times

Re: Last of Us Model Viewer/Extractor (PS4)

Post by shakotay2 » Sun Feb 19, 2017 8:01 am

[well, I've been told, my guess is wrong for PS4, does apply for PS3 models only]

Guess, it's also a matter of Sony's edge compression:
viewtopic.php?f=16&t=12070&p=99284#p99284

Only tool I know (so far) that can handle it is Noesis:
rapi.rpgCommitTriangles(edgeDecomp, noesis.RPGEODATA_USHORT, indexCount, noesis.RPGEO_TRIANGLE, 1)

But detailed informations are kept secret.
That's sad because more infos could help with Gran Turismo 6 Models, too.

I tend to ignore edge decompression, because it's so boring. I remember fiddling around with Sony's edge SDK (or something like that, not sure).
Just a waste of time and life... :D
Last edited by shakotay2 on Mon Feb 20, 2017 7:28 am, edited 1 time in total.
Bigchillghost, Reverse Engineering a Game Model: viewtopic.php?f=29&t=17889
extracting simple models: viewtopic.php?f=29&t=10894
Make_H2O-ForzaHor3-jm9.zip
"You quoted the whole thing, what a mess."

daemon1
M-M-M-Monster veteran
M-M-M-Monster veteran
Posts: 1761
Joined: Tue Mar 24, 2015 8:12 pm
Has thanked: 38 times
Been thanked: 1209 times

Re: Last of Us Model Viewer/Extractor (PS4)

Post by daemon1 » Sun Feb 19, 2017 12:32 pm

akderebur wrote:I did not, luxox is the guy you are looking for
luxox will not be able to help you. Hes just the guy who delivers decrypted files from the scene. He can't decrypt them.

akderebur
ultra-veteran
ultra-veteran
Posts: 324
Joined: Fri Jul 08, 2011 10:36 am
Has thanked: 44 times
Been thanked: 290 times

Re: Last of Us Model Viewer/Extractor (PS4)

Post by akderebur » Sun Feb 19, 2017 1:49 pm

Wobble wrote:Viewers and characters are nice, but as a coder, we would like to see some discussion on formats, algorithms, or code.
There is no real algorithm that can bring you further, it is just like pattern matching program designed specifically for The Last of Us pak files. Let me explain how the program works and you will see what I mean.

The way I did it was I basically opened bunch of character files in the hex editor and tried to find out recurring patterns in each file. Here is an example for getting vertex/index count and index offset.

There are 4 patterns for this (in hex) :
  • 03 00 00 00 00 00 00 00 00 00 00 00
  • 04 00 00 00 00 00 00 00 00 00 00 00
  • 05 00 00 00 00 00 00 00 00 00 00 00
  • 06 00 00 00 00 00 00 00 00 00 00 00
Also after 44 bytes from the start of the above patterns, there should be this pattern : FFFFFFFF.

So lets say the file contains 10 meshes. The program starts and searches for these patterns in the file and when there is a match it adds the offset to a list. When it is done there are like 3000 (I am making up numbers for this example) records in the list. It then goes through the list, for each offset it advances 44 bytes and checks if there is FFFFFFFF. If it finds out that pattern, it adds that offset (end of FFFFFFFF) – 56 bytes to a new list. The reason for offset - 56 bytes is because the data we are looking for actually starts there, again something that I found out looking at the hex editor. In the end that list has 10 records with correct offsets.

Now we can read the actual data. The program goes through each offset in the new list :
  • binary reader’s position is set to the current offset
  • first integer read = vertex count
  • second integer read = index count
  • advance 28 bytes
  • integer read = index offset
I think at this point you can see that the program is a product of binary search and pattern matching and that it doesn’t use a proper way to find data. It is like a bruteforcer for finding data in tlou files only. I find it hard to believe that anyone can develop it further to let’s say work with uncharted 4 files. If the file structure is identical, it will work. If not you would need new patterns and it would be a whole new program in that case.

The program as it is contains more code for graphics and ui programming than the actual code for reading the pak file and it is a bit messy :D. If you want I can strip the Unity stuff and share the part related to pak files.
shakotay2 wrote:Guess, it's also a matter of Sony's edge compression
Mesh data (index, vertex, uv) is not compressed in the ps4 files. Only the ps3 version has edge compression.

User avatar
Wobble
ultra-veteran
ultra-veteran
Posts: 585
Joined: Tue Jan 04, 2005 9:47 pm
Has thanked: 43 times
Been thanked: 112 times

Re: Last of Us Model Viewer/Extractor (PS4)

Post by Wobble » Mon Feb 20, 2017 12:37 am

[out]
Last edited by Wobble on Sat Mar 11, 2017 11:17 am, edited 1 time in total.

luxox18
mega-veteran
mega-veteran
Posts: 176
Joined: Fri Jul 29, 2011 9:18 pm
Has thanked: 54 times
Been thanked: 41 times

Re: Last of Us Model Viewer/Extractor (PS4)

Post by luxox18 » Mon Feb 20, 2017 2:22 am

ps4 developers are not using the edge compression anymore. Instead are using swizzled textures and models with same structure as PC with little endian data. (nothing compressed)

in this game specifically the vertex data and the uv data are separated and all faces start with 00 00 01 00 02.

not all games can be decrypted, for the moment is possible in games with firmware 1.76 (2013 - 2014 games)

oh, and about how to obtain decrypted games.. I just have a contact in the ps4 scene

User avatar
Wobble
ultra-veteran
ultra-veteran
Posts: 585
Joined: Tue Jan 04, 2005 9:47 pm
Has thanked: 43 times
Been thanked: 112 times

Re: Last of Us Model Viewer/Extractor (PS4)

Post by Wobble » Mon Feb 20, 2017 11:14 am

[out]
Last edited by Wobble on Sat Mar 11, 2017 11:17 am, edited 1 time in total.

Post Reply