Page 1 of 8

Megatextures research

Posted: Sun Apr 09, 2017 9:31 pm
by daemon1
Hello everyone.

After I did lots of research in sound formats, archives, models and animations I finally decided to get to most complicated task I've heard of: Megatextures.

This is current state of research: Doom/Evil Within/Wolfenstein tools are available here: http://aviacreations.com/wraith/#

And that's how it all started back in march'2017: after a couple of weeks I was able to reverse the modified version of HDP they used and decompress tiles. The devs improved the coding algorithm in compare to standard HDP.

Now I see that tiles are actually in correct order inside of megatexture as expected. This is just a random set of tiles taken from the middle of Wolfenstein Old Blood .pages file. I combined them manually, wanted to check the result as soon as possible ;)

Image

Re: Megatextures research

Posted: Mon Apr 10, 2017 10:53 am
by daemon1
Checked Doom. For each texture I'm getting 3 images 4 channels each. Not sure yet where is what, just got them for the first time, the whole process takes a minute. This is so slow, because its only a test tool now.

So I think this will work for all megatextured games.

Image
Image

Re: Megatextures research

Posted: Mon Apr 10, 2017 4:21 pm
by zaramot
This is an epic job! Thanks a lot. Especially for DOOM :)

Re: Megatextures research

Posted: Mon Apr 10, 2017 5:12 pm
by volfin
Good work. I could never get past the Codec, as that kind of thing was (and is) over my head. I look forward to seeing the final results. :)

As an aside, While Doom and Wolfenstein both use HDP, there's another game using Megatextures, The Evil Within. However, as far as I know, it's the only game that doesn't use HDP, They chose to use DCT instead.

enum pageCompression_t
{
COMP_INVALID = 0x0,
COMP_NONE = 0x1,
COMP_DXT = 0x2,
COMP_LZW = 0x3,
COMP_DCT = 0x4,
COMP_HDP = 0x5,
COMP_JXR = 0x6,
COMP_MAX_COMPRESSIONS = 0x7,
};

viewtopic.php?f=10&t=12110

Re: Megatextures research

Posted: Mon Apr 10, 2017 8:58 pm
by daemon1
Normal maps also look ok

Image

Re: Megatextures research

Posted: Mon Apr 10, 2017 11:41 pm
by luxox18
this is amazing!

your tool is able to extract a certain set of textures for a specific model or the entire vmtr, like the tool released by m0xf?

Re: Megatextures research

Posted: Tue Apr 11, 2017 7:14 am
by daemon1
My tool is able to extract anything in any order. You can get one texture of your choice, or ALL textures for one or all models or world. You can also grab the whole vmtr, which I don't see any useful, because it will be all in random order, why can anyone even need that?

You can surely extract the whole megatexture, or only the biggest MIP of it, with all tiles in correct order. But since many parts of megatexture are repeated, or not used (black squares), this texture will be even bigger than the whole vmtr.

Also, it will not require game exe, or running game, and will work with all games using HDP textures.

p.s. i have an idea. I can make a test to check if there are any parts of megatexture not used by any model. Like a search for hidden textures.

Re: Megatextures research

Posted: Tue Apr 11, 2017 12:28 pm
by luxox18
daemon1 wrote:My tool is able to extract anything in any order. You can get one texture of your choice, or ALL textures for one or all models or world. You can also grab the whole vmtr, which I don't see any useful, because it will be all in random order, why can anyone even need that?

You can surely extract the whole megatexture, or only the biggest MIP of it, with all tiles in correct order. But since many parts of megatexture are repeated, or not used (black squares), this texture will be even bigger than the whole vmtr.

Also, it will not require game exe, or running game, and will work with all games using HDP textures.

p.s. i have an idea. I can make a test to check if there are any parts of megatexture not used by any model. Like a search for hidden textures.
obviously that export a specific texture or a specific set of textures is more convenient than export the entire megatexture, this will help a lot, even in future games :D

I remember that the other tool doesn't export some pages from virtual textures like the alpha and the metalness pages (vmtrpower and vmtrcover) can you extract all pages, right? not only specular, diffuse and normal?

Re: Megatextures research

Posted: Tue Apr 11, 2017 12:45 pm
by daemon1
luxox18 wrote:I remember that the other tool doesn't export some pages
What do you mean by the other tool? I don't know any tool that can export Doom textures. Do you?

As for layers, I'm not sure what is metalness or power. But yes, I can export all layers.

Re: Megatextures research

Posted: Tue Apr 11, 2017 6:10 pm
by volfin
I think he means the tool for Wolfenstein. There was a tool by m0xf which hooked into the game to extract vmtrs.

Re: Megatextures research

Posted: Tue Apr 11, 2017 9:52 pm
by daemon1
volfin wrote:I think he means the tool for Wolfenstein. There was a tool by m0xf which hooked into the game to extract vmtrs.
I didn't check all layers in Wolfenstein yet. I was talking about Doom.

Re: Megatextures research

Posted: Tue Apr 11, 2017 10:18 pm
by luxox18
yes, I was referring about the wolfenstein tool but I was extrapolating that situation to the Doom textures.

also, thanks for this tool. finally a solution for obtain virtual textures directly and easily. :D

Re: Megatextures research

Posted: Wed Apr 12, 2017 5:09 am
by daemon1
luxox18 wrote:yes, I was referring about the wolfenstein tool but I was extrapolating that situation to the Doom textures.

also, thanks for this tool. finally a solution for obtain virtual textures directly and easily. :D
Sure :)

You don't have to extrapolate here. All games have slightly different formats. I'm checking other games meanwhile.

Re: Megatextures research

Posted: Wed Apr 12, 2017 10:13 am
by daemon1
Ok that DCT compression from Evil Within is much simpler than HDP. I've already cracked it. Here's some random part decoded from an_emergency_call_vmtr.pages, still need to browse megatexture correctly, but this will not take much time.

Note that color conversion may be incorrect, I just applied standard YUV to RGB.

Re: Megatextures research

Posted: Wed Apr 12, 2017 12:25 pm
by daemon1
yes, something may be wrong with colors. This is pl00 face, head, coat and cloth, which i believe must be the hero.

Image