Rainbow Six: Siege Models Thread

[killahtree]

The meshshape.forges are Havok data from what I remember, mesh.forge has the models.

All the forges got a reformat in March 2020, file headers changed. Compression and meshes should be the same though going back to when zaramot last poked his head in here.

Tushkan explained about the texture structures here: viewtopic.php?f=16&t=15031&start=120#p161589
He had a working texture parser that he didn't share, but it used RawTex and texconv.

He had a working model parser that he did share. No UI or executable, it was just a Python script you had to run manually but it did the job, full models with LODS and UVs, you can dig that up on his Dropbox if you're really interested. Not sure if it worked on post-March 2020 files out of the box, I haven't had the means to extract recent forges myself or I'd like to try that out.

His Dropbox: https://www.dropbox.com/sh/b2cuse4hp90y ... qTfja?dl=0

You'll want "export > outbreak_models.blend" because that includes the script head, but you'll also need to set up the "site-packages" in your Blender Python environment. Some extracted mesh files are also in the Dropbox to test on. You need to edit parts of the script to define the target to parse. Btw if you're a bit clueless and you're just looking for a tool where you can click some buttons and get the models you want this isn't it. No such tool exists so far.

Tushkan also had a working forge extractor, but did not share it and I'm not sure if he ever updated that one to work post March 2020. He said it wasn't hard to make that adjustment, but those format changes had interfered with it. The lack of a publicized forge extractor that worked properly with the current .forge file format in Siege was a barrier for us to be able to attempt the brute force approach with his model parser, but make no mistake, the brute force approach is rubbish. Having a tool that can extract a working model with good UVs isn't actually good enough, because what comes out of the forges is not ordered and has no useful names, so you'd be individually and laboriously processing files for hours, days, weeks to try and find the one mesh you wanted. It's not like there's just a .forge for the characters, and one for the guns. Instead consider it like there's like a hundred thousand assets thrown randomly into the forges, then you realize, it's no more reasonable than getting meshes off Intel GPA and painstakingly reconstructing the UVs and de-posing it. Without a way to search, it's a massive time investment to get anything you want. This is why Tushkan was also trying to solve the strings and dependency graphs, to be able to identify and target specific ones. That's one of the most important things we need to move forward on this. Tushkan was also trying to solve skeletons, weights, and so on, which can be nice but didn't help in him burn out on this project I guess..

The file name strings aren’t even used by the game, there is a good chance that they’re randomized or just a hash of the real file name.

[aajax]

The meshshape.forges are Havok data from what I remember, mesh.forge has the models.

All the forges got a reformat in March 2020, file headers changed. Compression and meshes should be the same though going back to when zaramot last poked his head in here.

Tushkan explained about the texture structures here: viewtopic.php?f=16&t=15031&start=120#p161589
He had a working texture parser that he didn't share, but it used RawTex and texconv.

He had a working model parser that he did share. No UI or executable, it was just a Python script you had to run manually but it did the job, full models with LODS and UVs, you can dig that up on his Dropbox if you're really interested. Not sure if it worked on post-March 2020 files out of the box, I haven't had the means to extract recent forges myself or I'd like to try that out.

His Dropbox: https://www.dropbox.com/sh/b2cuse4hp90y ... qTfja?dl=0

You'll want "export > outbreak_models.blend" because that includes the script head, but you'll also need to set up the "site-packages" in your Blender Python environment. Some extracted mesh files are also in the Dropbox to test on. You need to edit parts of the script to define the target to parse. Btw if you're a bit clueless and you're just looking for a tool where you can click some buttons and get the models you want this isn't it. No such tool exists so far.

Tushkan also had a working forge extractor, but did not share it and I'm not sure if he ever updated that one to work post March 2020. He said it wasn't hard to make that adjustment, but those format changes had interfered with it. The lack of a publicized forge extractor that worked properly with the current .forge file format in Siege was a barrier for us to be able to attempt the brute force approach with his model parser, but make no mistake, the brute force approach is rubbish. Having a tool that can extract a working model with good UVs isn't actually good enough, because what comes out of the forges is not ordered and has no useful names, so you'd be individually and laboriously processing files for hours, days, weeks to try and find the one mesh you wanted. It's not like there's just a .forge for the characters, and one for the guns. Instead consider it like there's like a hundred thousand assets thrown randomly into the forges, then you realize, it's no more reasonable than getting meshes off Intel GPA and painstakingly reconstructing the UVs and de-posing it. Without a way to search, it's a massive time investment to get anything you want. This is why Tushkan was also trying to solve the strings and dependency graphs, to be able to identify and target specific ones. That's one of the most important things we need to move forward on this. Tushkan was also trying to solve skeletons, weights, and so on, which can be nice but didn't help in him burn out on this project I guess..

I wasn't expecting a UI tool to just click buttons. I thought about a script or something like that. I study computer Science and hoped my knowledge could help me to get the models, but in the university you get trained to be a theorist. You have to learn the practics on your own. But everytime I try to research on the forge files or any I get some information that is not very useful or understandable for me. It's like everyone who knows something about it just gives small hints and then remains silent or just works on his own just to never share it to anyone. I hate to dependent in a to others in a way like that. But thanks for the information about that. I really appreciate it.

[Tushkan]

Sorry for my long absence, can't still find time to get back on track with this stuff. killahtree if you want to get hands on my code, you can ping me over at discord (Ulibos#3846) I'll share my sources and guide you through it. It's divided in 3 parts and needs some explanations to not get lost. I need someone to glance over it with extra pair of eyes to tidy some of it up and publish what I have so others can move on from where I left it.

As for your textures extractions: there are not only BC1 textures. Siege uses about 4 different types of dds encoding depending on texture type. Their forge versions are basically a shuffled vanilla dds with some strange decisions (like porting header to the end of file, storing dimensions not as pure w:h, but as w*num_channles:h*num_channels or smth like that). It would be easier for you to just grab my code and move on with it rather than rewriting that from scratch and losing all your steam in the process.

P.S.: you publish your discord id but your profile is locked for invitations, couldn't send a request to you xD

[Tushkan]

Ok boys and girls. It's 2021 and I think it's time to drop some news. I updated my code to work (more or less) with Y5 forges, made a couple scripts to ease the process and now I'm releasing it for you all to use. You can download it here https://www.dropbox.com/sh/b2cuse4hp90y ... qTfja?dl=0
There are readmes for you to be able to set it all up. I might have missed a thing or two while packing so if it breaks - don't panic, just post your problem here.
Also, I have no clue whether Ubi will break their forges again in Y6, so I'd HIGHLY suggest you to backup your current game distro. Would be a shame if next update would render my tool useless and you'd have to wait another 6 months for my patch, eh? )

[Custard]

Hooray! This looks very interesting

I wasn't able to find a module called binstream.py on the Dropbox.
Did found binstream as a pypi project, did pip install binstream, but it still fails the script on the first request made to it that doesn't seem like what you meant.

[Tushkan]

Oops, my bad. I dropped binutils instead of binstream. Updated dropbox now. Official binstream wouldn't work because mine is a custom wrapper that I use to write less code when deserializing. The only thing they have in common is their names ) Redownload the zip and you should be good to go.

[JakeMiles]

Very stupid question, are 4K Textures pack will be do diffirence and may use with unpacker?

[Tushkan]

I honestly never bothered to test. It looks like their uids are stored in asset files so they should be accessible via scripts. The only caveat is each texture package type (texture0,texture1,texture2,texture3) has it's own magic number which is used in r6s.tex.is_texture() to detect whether this file is to be parsed as texture. I suppose 4k textures should have their own file postfix like texture4.forge and have their own magic number for container. You need to add that magic into is_texture function for script to recognize those files correctly. You either can find it yourself and post it here, or I will need a sample file to find it myself. In readme there is a code to dump files from archive. You can dump just 1 file (replacing `for i,e,n,c in forge.files():` with `for i,e,n,c in forge[10]:`, it will only dump 10th entry from that forge archive) and send it to me so I can find and add that magic into the package.

[JakeMiles]

Thanks for reply, i'll try when can.

[JakeMiles]

Can you help me with some understand how to use it properly.
What i did:

I have installed Pytnon 3.7 (3.7.9)
install addon zstandard by this command:

Code: Select all

 pip install zstandard

create file in C:\Users\%username%\AppData\Local\Programs\Python\Python37\Lib\site-packages\siege.pth with this line:

Code: Select all

E:\SiegeUnpack\Tools

In this folder i have this:

Code: Select all

r6s\
rawtex\
scripts\
binstream.py
binutils.py

And i create folder E:\SiegeUnpack\Data for dumped files i think.
Then i edited E:\SiegeUnpack\Tools\r6s\settings.py
with this:

Code: Select all

IMPORT = (
    r"E:\Backup[G][Games]\Steam\steamapps\common\Tom Clancy's Rainbow Six Siege[upoff]"
)
#: path to dump uncompressed data to
EXPORT = r"E:\SiegeUnpack\Data"
#: binaries used for texture conversion
tex_bin = r"E:\SiegeUnpack\Tools\rawtex"

Okay then i will start simple check:

Step one:

Run(just double mouse click) E:\SiegeUnpack\Tools\scripts\roam_meshes.py
I will see UID and some numbers, okay, it will create folder "meshes\datapc64_merged_bnk_mesh\" with amount *.meshjson files.

Step two:
Run(just double mouse click) E:\SiegeUnpack\Tools\scripts\roam_textures.py
I will see something same, numbers UID etc. and it will create folder too. "textures\datapc64_merged_bnk_textures3\" with amount *.png files.

Well like all good? Yes i think.

Step three:

Run again double clicked E:\SiegeUnpack\Tools\scripts\dump_asset.py
And folder assets didn't create and nothing happend window just closed, on this step i think a do something wrong.

I missing some instructions in readme file? i've glad if you help me what i do wrong.

P.S. Game install without 4K textures pack, just standart install.

[JakeMiles]

And in readme said about cache, i don't have it, how can i create him and how i need it, for easely asset export?
I notice roam_meshes.py export only guns, when i comment this line:

Code: Select all

       # filter meshes that most likely belong to guns
                # if (
                #  mesh.x2C not in (9,10)
                # ): continue

it start dump every mesh, how can be possible to know type of meshes, like, guns, character and etc. i'll be happy to dump some character if it possible.
If i understand right it possible only by asset file, wich i can't get.

[Tushkan]

Cache is created automatically by the scripts that need it. You should see a folder named cache inside scripts dir. I can't tel what went wrong without seeing dump_asset's console output. DOn't run it via double-click. Run cmd.exe, navigate to the scripts folder, then run the script in that cmd. It won't close after executing script and you will be able to see error report. I need that report to diagnose what went wrong.

If you want to search for specific types of objects, you will have to read r6s source. I comment all parameters that I reverse. So you need to open r6s/mesh/__init__.py and read class Mesh definition. There are different parameters that allow to partially filter needed data.

[JakeMiles]

Here the error msg:

Code: Select all

C:\Windows\system32>E:

E:\>Cd E:\SiegeUnpack\Tools\scripts

E:\SiegeUnpack\Tools\scripts>dump_asset.py
loading E:\SiegeUnpack\Tools\scripts\cache
generating None
Traceback (most recent call last):
  File "E:\SiegeUnpack\Tools\scripts\dump_asset.py", line 43, in <module>
    asset_links = data.asset_children.child_in_links(UID)
AttributeError: 'NoneType' object has no attribute 'child_in_links'

[Tushkan]

It didn't find any asset that would have given UID as a child. So it returned None instead of list. Loop couldn't iterate through None object and failed. In short, no asset found that has that id as a child. How do you insert a UID you're truing to search?

[JakeMiles]

Its default UID from dump_asset.py UID = 22439848753, or you mean in need place files to folder wich i self create for this asset number?