Join also our Discord channel! Click here.

Rainbow Six: Siege Models Thread

Post questions about game models here, or help out others!
User avatar
Custard
beginner
Posts: 23
Joined: Fri Oct 25, 2019 12:33 am
Has thanked: 7 times
Been thanked: 2 times

Re: Rainbow Six: Siege Models Thread

Post by Custard » Sat Mar 07, 2020 11:53 am

Yup, setting submesh number to zero solved it right away, it's putting out proper meshes. :]

What console do you mean? if OS' cmd interface, then it won't run there because import script relies on bpy module which is strictly blender-specific. If in blender's python console, then it should run if all script is copy-pasted there, including imports etc (IIRC "Run Script" button and blender console run in different scopes, i.e. they don't share imports or variables). Hitting "Run Script" is actually the way to go.
I meant Blender's internal python console, sorry wasn't clear. It seemed like an import issue since it was failing due to functions "not existing", but if I'm understanding you right that code doesn't need to be used anyway, when running test_mesh_parms does the job instead.

Examining Mesh class I can see a fair bit of WIP in there. If I wanted to print out a mesh header to view, is that accommodated for?

Attempting to parse files unpacked by the Delutto unpacker throws a decode error:
  • File "S:\Tools\Blender\2.82\python\lib\site-packages\r6s\common.py", line 8, in __init__
        self.name = r.read(name_len).decode('utf8')
    UnicodeDecodeError: 'utf-8' codec can't decode byte 0x95 in position 8: invalid start byte

I got the same outbreak .forge from comword's dropbox and extracted and processed a mesh file that was the same as one I already had from your dropbox, all I could tell was the contents of the two were not the same. Guessing it's either output bad data, or ins't compatible in some other way. Not much incentive to continue with that path as you've solved unpacking separately, unless you got some simple idea that fixes it.

Tushkan
advanced
Posts: 59
Joined: Mon Dec 18, 2017 1:47 pm
Has thanked: 1 time
Been thanked: 15 times

Re: Rainbow Six: Siege Models Thread

Post by Tushkan » Sat Mar 07, 2020 12:10 pm

My tool expects a raw uncompressed file data from forge. These files always have a container wrapper around them that has their magic, size and optional string and then uid. The failing code from Common parses that data. Now there are the following options:
1) denutto's unpacker unpacks data and rips away that header. So common tries to read actual mesh data as utf-8 string and fails.
2) denutto's unpacker is up to date with recent file structure change (where they now store packed metadata in that wrapper instead of string) so my wrapper reader tries to interpret packed meta as a utf8 string and fails. I need a file sample in order to get to the bottom of the problem. You can share it somehow or again PM me the discord id, that will make turnarounds faster.

User avatar
Custard
beginner
Posts: 23
Joined: Fri Oct 25, 2019 12:33 am
Has thanked: 7 times
Been thanked: 2 times

Re: Rainbow Six: Siege Models Thread

Post by Custard » Sun Mar 08, 2020 8:54 am

The version of the tool I got is from late 2018, I'm not aware of a later version and it's not being developed anymore (and was never really considered finished at the time). Now that you mention it, neither this version of your parser or this unpacker tool are going to work right with my current version of the game, which is what I was planning on investigating those compression types on. Or are those structure changes only in the TTS right now? I might have a few days to save a copy.

I put two Delutto unpacked files in dropbox.
9120 is that helicopter mesh from that outbreak forge, I just picked something that could be compared.
2231 is from current live version of the game — datapc64_merged_set02_bnk_meshshape.forge from which you indicated compression issue. If you can tell if this has been properly decompressed it means it's probably using lzo then. His tool also uses Oodle library, bit of a questionmark for me there trying to find out about it but I think it's used for image compression.

https://www.dropbox.com/sh/gusxvns2h6j2 ... pdQGa?dl=0

And yeah it outputs a weird file extension that makes it look like a script error..

Thanks for the Discord offer. Slow pace is suiting me at the moment, I hope not to be inconvenient to you.

Tushkan
advanced
Posts: 59
Joined: Mon Dec 18, 2017 1:47 pm
Has thanked: 1 time
Been thanked: 15 times

Re: Rainbow Six: Siege Models Thread

Post by Tushkan » Sun Mar 08, 2020 10:17 am

Tts has the updated storage format. Current official build (that has wamai an kali introduced) still has the old format that I reverse. So yes, you can grab a copy of it. My parser should be easily adjusted for new forges except for metadada. I still don't know how they pack it but I can simply skip it.
I'll take a look at your sample soon, thanks.
Discord is alot more handy to my liking because I always have it online and it supports small file and image sharing. But forum is more transparent for people in terms of following the progress so I won't insist )

Edit: Oh, it's easy ) Denutto's unpacker works. You just unpack the wrong forge ) meshshape.forges store havok data. r6s.mesh expects data from mesh.forges so you have to unpack datapc64_merged_set02_bnk_mesh.forge And ignore the file formats. he uses magic as a file format, he just seems to parse it as int instead of uint so you get the minus. If you convert -1842221553 back to bytes, you will get 0F EE 31 92 which is the magic number for havok files. This also means last version of Denutto uses zstd, not lzo. Otherwize it wouldn't be able to unpack the data.

Grab a sample from mesh.forge and put it onto dropbox. I still have suspicion that he trims the header of a binary. If that's the case, you'll have to make a slight modification in r6s.mesh to make it work.

Tushkan
advanced
Posts: 59
Joined: Mon Dec 18, 2017 1:47 pm
Has thanked: 1 time
Been thanked: 15 times

Re: Rainbow Six: Siege Models Thread

Post by Tushkan » Sun Mar 08, 2020 6:47 pm

Updated site-packages and .blend scene. In theory it should import more stuff correctly. But right now I only tested it on a Smasher (1830.mesh). Imports all islands for given lod, assigns materials slots per island, assigns uv. Any unknown data is mapped onto vertex colors. Right now there are 4 or 5 channels for smasher, I guess those are skin weights but I can't seem to grasp how they are packed. mesh object now also contains additional metadata per island. I can parse it but I can't understand what it's meant for in some cases (for example Mesh.island_bboxes is a bad name, it's not a bbox). I have found some other extra data which I will post later, maybe someone here will be able to solve it.

EDIT:
Fixed a bug with zero length islands that would lead to an exception. If anyone has downloaded new script, redownload mesh\__init__.py. Be aware, there might still be plenty of bugs.
full_mesh_import.jpg
You do not have the required permissions to view the files attached to this post.

comword
ultra-n00b
Posts: 7
Joined: Thu Nov 22, 2018 10:38 am

Re: Rainbow Six: Siege Models Thread

Post by comword » Mon Mar 09, 2020 5:18 pm

Tushkan wrote:
Sun Mar 08, 2020 6:47 pm
Updated site-packages and .blend scene. In theory it should import more stuff correctly. But right now I only tested it on a Smasher (1830.mesh). Imports all islands for given lod, assigns materials slots per island, assigns uv. Any unknown data is mapped onto vertex colors. Right now there are 4 or 5 channels for smasher, I guess those are skin weights but I can't seem to grasp how they are packed. mesh object now also contains additional metadata per island. I can parse it but I can't understand what it's meant for in some cases (for example Mesh.island_bboxes is a bad name, it's not a bbox). I have found some other extra data which I will post later, maybe someone here will be able to solve it.

EDIT:
Fixed a bug with zero length islands that would lead to an exception. If anyone has downloaded new script, redownload mesh\__init__.py. Be aware, there might still be plenty of bugs.
full_mesh_import.jpg
I didn't see same result in .blend as the photo, which is much more smooth and complete than export/outbreak_models.blend, also this monster is called Apex. I guess some of unknown data is reflection or diffuse or light emission. You can see some parts of their body are red glowing. In addition, have you tried to unpack textures?
Image

Tushkan
advanced
Posts: 59
Joined: Mon Dec 18, 2017 1:47 pm
Has thanked: 1 time
Been thanked: 15 times

Re: Rainbow Six: Siege Models Thread

Post by Tushkan » Mon Mar 09, 2020 6:39 pm

I know that's apex ) I used him to test for a bug and I added him after I edited the original message.
If you use latest scripts, your imports must also contain multiple isnands per char and smooth normals.
Glow is most likely stored in textures, not vertex color. Judging by Apex' fingers those are most likely skin weights. I just need to figure out the packing.
Yes, I can unpack textures. Main issue is tracking the ones that belong to an asset. For that I need to turn back to reversing asset structures.

User avatar
Custard
beginner
Posts: 23
Joined: Fri Oct 25, 2019 12:33 am
Has thanked: 7 times
Been thanked: 2 times

Re: Rainbow Six: Siege Models Thread

Post by Custard » Tue Mar 10, 2020 7:43 am

Tushkan wrote:
Sun Mar 08, 2020 10:17 am
Edit: Oh, it's easy ) Denutto's unpacker works. You just unpack the wrong forge ) meshshape.forges store havok data. r6s.mesh expects data from mesh.forges so you have to unpack datapc64_merged_set02_bnk_mesh.forge And ignore the file formats. he uses magic as a file format, he just seems to parse it as int instead of uint so you get the minus. If you convert -1842221553 back to bytes, you will get 0F EE 31 92 which is the magic number for havok files. This also means last version of Denutto uses zstd, not lzo. Otherwize it wouldn't be able to unpack the data.

Grab a sample from mesh.forge and put it onto dropbox. I still have suspicion that he trims the header of a binary. If that's the case, you'll have to make a slight modification in r6s.mesh to make it work.
Oh, I seem to have misunderstood something. When you posted that list prior.. uh, this one, I thought that was pointing to the location of some files that you weren't able to decompress with zstd. I didn't actually try to put the meshshape file through the parser, I included that one because it was on that list of yours. I thought that file had the unknown compression type. The other file (9120) was proper mesh file and was the one I got the error on when attempting to parse. Delutto's tool does use zstd, but lzo and oo2core as well. Where it's using those libraries is another matter.

blenderprscr2.jpg
Just because it was in the view, this monster seems to have suffered localized black hole to the groin. I expect you already noticed that. But on the topic of UVs, this game is using UDIMS? I see a lot to suggest that it uses multiple UV maps per mesh for certain objects, whether that's actual separated UV maps or they are suppose to all be on UDIMs I dunno. But for example when zaramot got Ash he seemed to export all of Ash body from one mesh file, and all of Ash head from another. However, from the operators I have from Luxox I can see that operators have far more texture sets which do not co-exist on the same UV tile:

Ash
  • Eyelashes
  • Hair
  • Head
  • Hands
  • Torso
  • Arms
  • Pants
  • Boots
  • Badges
IQ
  • Eyelashes
  • Eyes
  • Hair
  • Head
  • Head_det (some blank texture)
  • Helmetvisor
  • Holster
  • Gear
  • Torso
  • Arms
  • Arms_det (a tileable)
  • Arms_det2 (a tileable)
  • Legs
I see overlapping UVs for the monster too, could it be what you've been calling sub-meshes is where UVs are separated by?
You do not have the required permissions to view the files attached to this post.

Tushkan
advanced
Posts: 59
Joined: Mon Dec 18, 2017 1:47 pm
Has thanked: 1 time
Been thanked: 15 times

Re: Rainbow Six: Siege Models Thread

Post by Tushkan » Tue Mar 10, 2020 9:40 am

The lack of groin seems to be intentional, not a parser error. Maybe that's why he's so angry ))
New script supports exporting separate islands, you can add islands=%island number% to mesh.build_meshdata, it will only build the specified island (smasher has 5 of them). If you build only the first island, you will notice there are no uv overlaps and the part that goes beyuond u=1 is actually almost identical to the left one. They unwrapped left and right parts of body identically and placed them in different uv quads for ease of management. But the texture wraps around the edge and tiles so you basically have them uv-ed identically, like if they where stacked, no udims involved.
In fact they use 1 material per island, that's why they consume uv space completely per each island. If you look into material tab in blender for that mesh, you will notice it creates meshslots per each usland so you can apply different materials on them like they do it in the game itself.
Smasher's head is also in different file. It uses only 2 uslands (actual head and a neck hole filler). On characters you will have more islands because hair, eyes, eyelashes - all those require different shaders (I can bet each island even gets rendered in a sepatare render call).

Tushkan
advanced
Posts: 59
Joined: Mon Dec 18, 2017 1:47 pm
Has thanked: 1 time
Been thanked: 15 times

Re: Rainbow Six: Siege Models Thread

Post by Tushkan » Tue Mar 10, 2020 10:01 am

These metas where sampled from TTS files. It's not just those that are unknown. It's ALL metas are packed like that in TTS. I just randomly sampled about a 100 of them from different files to get some diversity in data. Those need to get piped through different decompressors in order to try to identify the right one. I just can't find time for that so I rely on community's help on that one.

User avatar
Custard
beginner
Posts: 23
Joined: Fri Oct 25, 2019 12:33 am
Has thanked: 7 times
Been thanked: 2 times

Re: Rainbow Six: Siege Models Thread

Post by Custard » Tue Mar 10, 2020 3:30 pm

You're making a large amount of sense, thank you senpai. :D

They were NOT kidding about those permeating data changes huh, new season patch has gone live and it's 105 GB for me. That's pretty much the entire game. Actually, I think it's more than the entire game and includes the optional high-def texture pack which I have installed as well, which I assume lives among the .forges.

I encountered an error when playing around with new script, a file which was parsed before now fails. This is the number 12723.mesh, containing the pretty awesome tilt rotor craft if I do say so. I iterated through the islands using the parameter you suggested — mesh.build_meshdata(islands=%island number%) — The mesh has 15 islands and all of them parse corrected except number 8, which judging by the other islands I seen number 8 must be the main body of the aircraft. Here is the log of my iterations and the error:

Code: Select all

=================
remaining 0
built data
1
^^^^^
12723
=================
remaining 0
built data
2
^^^^^
12723
=================
remaining 0
built data
6
^^^^^
12723
=================
remaining 0
built data
7
^^^^^
12723
=================
remaining 0
built data
8
^^^^^
12723
=================
remaining 0
built data
11
^^^^^
12723
=================
remaining 0
built data
13
^^^^^
12723
=================
remaining 0
built data
Rebuilt loops from 4764 to 4755.
Array iterator out of range: (index 1585)
Traceback (most recent call last):
  File "S:\R6SRip\Scripthead2.blend\test_mesh_parms", line 68, in <module>
  File "S:\R6SRip\site-packages\r6s\blnd.py", line 106, in build_dict_mesh
    mesh.polygons[poly_id].material_index = mat_id
IndexError: bpy_prop_collection[index]: index 1585 out of range, size 1585
Error: Python script failed, check the message in the system console

=================
remaining 0
built data
5
^^^^^
12723
=================
remaining 0
built data
14
^^^^^
12723
=================
remaining 0
built data
9
^^^^^
12723
=================
remaining 0
built data
12
^^^^^
12723
=================
remaining 0
built data
10
^^^^^
12723
=================
remaining 0
built data
3
^^^^^
12723
=================
remaining 0
built data
15
^^^^^
12723
=================

User avatar
Custard
beginner
Posts: 23
Joined: Fri Oct 25, 2019 12:33 am
Has thanked: 7 times
Been thanked: 2 times

Re: Rainbow Six: Siege Models Thread

Post by Custard » Tue Mar 10, 2020 3:59 pm

Tushkan wrote:
Tue Mar 10, 2020 9:40 am
The lack of groin seems to be intentional, not a parser error. Maybe that's why he's so angry ))
Haha yes I remember he was very angry. Though, in the interest of avoiding misunderstanding due to my off-hand comment, in the image on the right-hand leg I am referring to mesh deformity, badly stretch faces causing a ugly shading there. I can believe it if the error was part of the mesh and not your parser, but sometimes I'm not very clear, just ignore this if that's what you meant all along.

Tushkan
advanced
Posts: 59
Joined: Mon Dec 18, 2017 1:47 pm
Has thanked: 1 time
Been thanked: 15 times

Re: Rainbow Six: Siege Models Thread

Post by Tushkan » Tue Mar 10, 2020 5:13 pm

Yes, you got 100Gb update because of extra textures. My friend had just reported that it's 63Gb for him. They have essentially reuploaded all forges. Be aware that Denutto's tool in theory should be able to unpack new forges (mine does, it only messes up some indexing meta), but my scripts won't be able to parse these meshes yet (pesky new metadata format).
Now for your error message. There is a message "Rebuilt loops from 4764 to 4755." This happens because I invoke blender's internal validation. For some reason it crops off some of the triangles, causing the vertex count to shrink. That's why you get IndexError. If you want to bypass it, set validation=0 in build_mesh, it will skip the pass and should load normally then. I'll try to fix it later.

User avatar
GordenF
n00b
Posts: 16
Joined: Fri Jul 15, 2016 11:14 pm
Has thanked: 2 times
Been thanked: 2 times

Re: Rainbow Six: Siege Models Thread

Post by GordenF » Wed Mar 11, 2020 12:33 pm

I just found this thread via google and was pleasantly surprised that someone was already working on this game.
Is it possible to extract operator models with their skeletons for simple posing? All the operator models I could find online are boneless, which makes them useless for anything beyond simple renders.

Tushkan
advanced
Posts: 59
Joined: Mon Dec 18, 2017 1:47 pm
Has thanked: 1 time
Been thanked: 15 times

Re: Rainbow Six: Siege Models Thread

Post by Tushkan » Wed Mar 11, 2020 2:46 pm

GordenF wrote:
Wed Mar 11, 2020 12:33 pm
Is it possible to extract operator models with their skeletons for simple posing?
This is basically my goal for this project. I'm yet to extract skeletons. Zamarot was able to pinpoint where this data is stored, but he is not that interested in siege, so he handed me the hints on what to search. Right now my plans are to ease up dds extraction and then I will proceed with skeleton data reversing.

Post Reply