Join also our Discord channel! Click here.

Unreal engine 4 uncooking maps

The Original Forum. Game archives, full of resources. How to open them? Get help here.
Post Reply
123185321f
beginner
Posts: 26
Joined: Sat Apr 28, 2012 1:53 am
Has thanked: 6 times
Been thanked: 4 times

Unreal engine 4 uncooking maps

Post by 123185321f » Wed Apr 22, 2020 5:45 am

This is a redirect from https://www.gildor.org/smf/index.php/to ... l#msg35982 to see if this community has any suggestions about improving the process (I'm the OP there) :D

//copy/paste
tldr;
I have very very good news! and some less than very good news! hahaha!

The very good news is that I managed to automatically re-create in the editor a simple map from a cooked map.
The less than very good news is that it works for very simple maps so far, complex maps (even interior maps without terrain) still have some serialization issues.

But hey, at least we proved a simple proof of concept! hehehe

An image example:
https://drive.google.com/open?id=1WgCt4 ... VhhSrS6DRm

details

Code: Select all

ULevel::Serialize
was instrumental for reverse engineering!
In my current tests I switched the default editor map to point to the cooked map, and juggle the stream into a functional linker.

But even in such a simple map, streaming the cooked map data was not perfect, for example the brush component for the lightmass importance volume was not properly un-serialized, so the end result has an empty lightmass importance volume that is basically useless. But the light and the floor were correctly un-serialized, which was my goal anyways.

In my tests I also tried the Sanctuary ship from Borderlands 3 which is an interior map (I think! :D). But I encounter some serialization problems, so I have to poke around some more. The good news was that out of ~11,000 object, about 9000 of them were correctly identified, so ~80% of the map was at least somewhat serialized, albeit not perfectly. I'm still working on this, one of my current problems is that the serialized offset is reported wrong, I don't currently know why :-)

But I'll poke around and beat it to submission. Or just fail :D :D. Yay me and my self-deprecation of Average Joe-ness! hahaha!

TrumpetPro
beginner
Posts: 39
Joined: Wed Jul 06, 2016 1:51 am
Has thanked: 1 time
Been thanked: 1 time

Re: Unreal engine 4 uncooking maps

Post by TrumpetPro » Mon Apr 27, 2020 10:16 pm

Nice work! Any plans on releasing your tools, even it's current state?

123185321f
beginner
Posts: 26
Joined: Sat Apr 28, 2012 1:53 am
Has thanked: 6 times
Been thanked: 4 times

Re: Unreal engine 4 uncooking maps

Post by 123185321f » Wed Apr 29, 2020 12:08 am

Follow-up to automatically restoring cooked maps back in unreal engine 4, this is the latest successful test

Borderlands 3 Sanctuary Map automatically Uncooked Test:
https://www.youtube.com/watch?v=6AhRN6vd7t8

Image


The heavy lifting is done in

Code: Select all

ULevel::Serialize
in

Code: Select all

Ar << Actors;
line in the unreal engine source code

123185321f
beginner
Posts: 26
Joined: Sat Apr 28, 2012 1:53 am
Has thanked: 6 times
Been thanked: 4 times

Re: Unreal engine 4 uncooking maps

Post by 123185321f » Tue May 05, 2020 12:42 am

Test 2. With Lilith :D :D

click on Link!
https://youtu.be/jk6xjHVx2-o

Image

Kein
advanced
Posts: 56
Joined: Fri Nov 06, 2009 12:57 pm
Been thanked: 5 times

Re: Unreal engine 4 uncooking maps

Post by Kein » Fri May 15, 2020 9:21 pm

You alive, m8? I actually wanted to talk about your progress, PM me if your heart still beats!

123185321f
beginner
Posts: 26
Joined: Sat Apr 28, 2012 1:53 am
Has thanked: 6 times
Been thanked: 4 times

Re: Unreal engine 4 uncooking maps

Post by 123185321f » Tue Jun 16, 2020 8:01 am

the first attempt was kinda simplistic, in the sense that I learned I need to enable two options in unreal engine in order to handle directly cooked assets
1. Requires unreal engine 4.22+, because they implemented ...
2. 'Allow cooked content in the editor' Starting with 4.22
3. 'Allow unversion content in the editor' in UObjectGlobals.cpp

Code: Select all

int32 GAllowUnversionedContentInEditor = 1;
After this was done, Level.cpp in

Code: Select all

void ULevel::Serialize( FArchive& Ar )
the line

Code: Select all

Ar << Actors;
Populates most of the default actor types, such as lights, static actors and so on.

Currently, landscape component fails to import using this approach.

I also did some tests using my approach with Jedi fallen order, because as it happens the level I was looking at didn't use any landscapes, which made it easier to import.

But one of my problems is actually materials, in particular cooked materials, although I can easily find the material expressions/functions, I don't know currently the connections map. That is I don't know, I couldn't find a pattern, which material expression connects to which node.
Which of course leads to a transitivity problem, because (2+2)*3 != 2+(2*3). Although I correctly identified the twos and the threes :D

123185321f
beginner
Posts: 26
Joined: Sat Apr 28, 2012 1:53 am
Has thanked: 6 times
Been thanked: 4 times

Re: Unreal engine 4 uncooking maps

Post by 123185321f » Tue Jun 16, 2020 8:01 am

100% cooked ingredients!
cooked sounds, cooked materials, cooked static meshes, cooked map, etc., all reimported in unreal directly from Heroes Showdown

Image

If the YouTube video doesn't display, here is the direct link
https://www.youtube.com/watch?v=bxFRdel2cLw

A few of us made a small discord channel for this
https://discord.gg/hyC5qgU

It is mostly for discussions and tests, this is not meant to be a full, easy to use tool...
Should you join, browse the previous comments in the discord channel for some extra relevant information
Complex landscape still doesn't work in cooked form though...

123185321f
beginner
Posts: 26
Joined: Sat Apr 28, 2012 1:53 am
Has thanked: 6 times
Been thanked: 4 times

Re: Unreal engine 4 uncooking maps

Post by 123185321f » Wed Jun 24, 2020 2:33 am

Inspired by Daemon's success in getting the landscape from some unreal engine 4 games, I also looked into this a bit, and I think I found a general pattern that hopefully applies to all?/most?/some? unreal engine cooked maps.

I'll explain the process as I understood it by looking at the data I extracted from a map from Heroes Showdown that contains landscape ( my example above does not contain landscape, only static meshes ). My explanations may not be scientifically accurate or use the scientific correct language, but I'm hoping to gist of it is correct :D

When the landscape is being cooked, unreal creates a grid of landscape components based on the settings used when the landscape was generated in the first place in unreal editor. I will stick to the Heroes Showdown map example, as it's easier

Image

So in my case, the landscape was broken into 324 landscape components = 18^2
Also, the complete Height map texture, would also be broken down into chunks, separate textures, each texture Chunk assigned to each component.
All this is being cooked together.

When uncooking, I found myself in the unhappy situation of having a bunch of very small textures chunks that I had to find a way to glue back together like a puzzle. Luckily the X and Y location in the atlas? of the height map is being cooked into each landscape component.

So then I just had to basically get each textures chunk and added to an empty canvas at the correct X and Y, like so (In my case at least, the actual Y was Mirrored, from bottom to top, e.g. if cooked Y=14 and Y Max=252, the final Y=YMax-14=238)
Image

Then eventually I got a complete height map (I used C++ CImg library for appending the chunks into one complete 2-D Texture)
Image

Which I Imported in unreal Editor, and generate a landscape based on the Setting and Height map Extracted from the cooked content, here is a bird eye view Of the final generated landscape in unreal editor, which is remarkably similar to the height map glued together from the texture chunks extracted from the cooked content
Image

Eventually, I may try to apply the same pattern to a different unreal engine cooked map from a different unreal engine game, but for now I'm interested in learning more about foliage, so that's my next stop! :D

123185321f
beginner
Posts: 26
Joined: Sat Apr 28, 2012 1:53 am
Has thanked: 6 times
Been thanked: 4 times

Re: Unreal engine 4 uncooking maps

Post by 123185321f » Fri Jul 03, 2020 4:39 am

Another landscape example, this time Borderlands 3 Towers, with an attempt for some basic materials

Bird eye view
Image

Landscape generated from this height map
Image

Landscape materials automatically blended according to this weight map
Image

Weight map blending close-up example.
Image

Both the height map and the weight map were extracted from Borderlands 3 cooked assets.
Note I'm a scientist not an artist, although I sometimes understand serialized zeros and ones, I don't have any experience with actual technical artist/designer field, so I have no idea if the end result is correct :D

Post Reply