XeNTaX

Has anyone ever used 123D and gotten something out of it? Like with game screenshots or something.

AceWell wrote:If you can get enough good images from video then you could use 123D Catch to create a 3d model from them.

Or, you can actually reverse-engineer a model format, so you can get bones and weights, which would be the SMART way to do it if you want models to actually use ...

And I should mention, a moderator that shall remain un-named pointed out this thread to me, and we both got a good chuckle over it all, so I decided to post my first response to his question.

it wouldn't be practical to take a ingame screenshot... this only works on simple objects, with good lighting. Not to mention you have to draw control points so the thing doesnt turn into poop

brienj wrote: Or, you can actually reverse-engineer a model format, so you can get bones and weights, which would be the SMART way to do it if you want models to actually use ...

What if I just wanted to just turn my entire city into a game map with a series of photos?
Definitely nothing to reverse and it'd be a pain to create manually

the topic is about console extraction.. LOL I would love to see how you'd convert a city/landscape from the screenshots of any current gen game.

the dynamic shadows would make it impossible and make any conversion turn into a bumpy mess

finale00 wrote:

brienj wrote: Or, you can actually reverse-engineer a model format, so you can get bones and weights, which would be the SMART way to do it if you want models to actually use ...

What if I just wanted to just turn my entire city into a game map with a series of photos?
Definitely nothing to reverse and it'd be a pain to create manually

It's not practical at all, especially on a console.

brienj wrote:Or, you can actually reverse-engineer a model format, so you can get bones and weights, which would be the SMART way to do it if you want models to actually use ...

or you could just create whatever you want from scratch, but neither reverse engineering nor scratch making models is within the scope of this thread.

Derpy derp??



It's a little bit imposible...

mariokart64n wrote:it wouldn't be practical to take a ingame screenshot... this only works on simple objects, with good lighting. Not to mention you have to draw control points so the thing doesnt turn into poop

Practical or not, if the author of the thread wants to connect the console to the PC to get models then he's running out of options. Since the actual rendering is being done on the console and is only displayed on the PC as video then camera tracking and taking ingame images through free camera features and turning them into 3d objects are his choices. If he wants the models bad enough then he'll have to do whatever it takes within his abilities to acquire them. An image is an image whether taken in real life or in a game, many newer games have awesome graphics these days anyway (ex: Crysis 2), a game image could be cleaned up in photoshop etc if need be. I'll play around with this stuff when i get some time and see what kind of quality can come of it.

Gentlemen, gentlemen! 't-is but a scratch!

We can conclude that the original question is an interesting one in terms of technological implications. FaceGen is one of those things that can capture a face and turn it into a 3d model. You still have to tweak the result. The tool is dedicated and optimized for still images of faces. There are probably other similar tools out there.

In any event, we can also conclude that the best way to move forward would be to figure out the software format of the models, and in that respect we have quite a number of members here that could possibly assist in that.

As for pun and jokes at the expense of others, I can only say the famous quote of Lo Wang after having slain another martial arts opponent: Welcome to real life, Mr. Chan!

All had their say, now let's move on.

It's actually a bit interesting to think of how this could be somewhat practical when combined with a dump of the depth from the main backbuffer. (assuming the game doesn't use a separate framebuffer for depth, which a lot of games do particularly to achieve linear depth and use that for fog/screenspace shadows/etc., then dumping it is still possible but a good bit trickier) If you could further get the MVP matrix used in the draw you could do a lot more (like transforming the fragment coordinates by the inverse projection to get clipped but still reasonably exact modelspace coordinates), but if you can get that then you might as well just be dumping the vertex data directly. So unless you're in a situation where you can screenshot the depth buffer but for some reason can't access anything else in shared memory, the whole approach is pretty useless. If you are in that situation... it's still pretty useless but could manage to net you some scraps of usable geometry.

MrAdults wrote:It's actually a bit interesting to think of how this could be somewhat practical when combined with a dump of the depth from the main backbuffer. (assuming the game doesn't use a separate framebuffer for depth, which a lot of games do particularly to achieve linear depth and use that for fog/screenspace shadows/etc., then dumping it is still possible but a good bit trickier) If you could further get the MVP matrix used in the draw you could do a lot more (like transforming the fragment coordinates by the inverse projection to get clipped but still reasonably exact modelspace coordinates), but if you can get that then you might as well just be dumping the vertex data directly. So unless you're in a situation where you can screenshot the depth buffer but for some reason can't access anything else in shared memory, the whole approach is pretty useless. If you are in that situation... it's still pretty useless but could manage to net you some scraps of usable geometry.

It could be even more practical if there were a way to allow the PC to do the rendering, and interrupt it with a 3rd party which automatically extrapolates pertinent information from the RAM... And I realize this may sound like a trolling attempt, but I did once rip models with 3DVia Printscreen, in their bind pose, as opposed to the shape they were in on screen.

I think there will eventually be something that can capture running memory and make sense of it... however, it will likely not be for another ten years, and even then, there is no technology existing right now that allows a live transfer of total memory from a console to a PC.

However, what MrAdults says is intriguing I admit. It would be something along the lines of converting a normal map back into a highpoly model, since normal maps use individual channels of red, green, and blue, to represent x, y, and z difference.
I think the only problem with MrAdult's interesting concept is that the depth buffer would technically omit backfacing and obscured polygons (like any hidden behind another polygon or object,) would it not? That would leave you with half a model properly formed, and half flat, or omitted entirely... Then again that's assuming the depth buffer is as I understand it to be.

Normal maps don't contain a length, they're representing unit vectors, so deriving an actual shape from them would be completely just guess work without height/distance information. A depth buffer gives you a length, but yes, only on the projection axis.

If you had a whole system RAM dump, getting information out of that automatically isn't a 10-years-down-the-line problem. If you make the initial assumption that positions are as 32-bit floats, you can scan the entire memory for valid floating point values. If you find 3 or more in a row, you then scan nearby memory. If you have a giant block of them you may have a vertex buffer, or if you have segments of 3 or more valid floating point values on a fixed stride apart then maybe you've found an interleaved vertex buffer. Then you can try interpreting those values in a variety of ways. You can tally them up and pay attention to whether you've got a flat block of positions or an interleaved buffer to try to determine the total number of vertices, and use that knowledge to even further scan for 16-bit or 32-bit index buffers that fall into a range you expect based on your counts. Or you can rely on one of the many semi-reliable algorithms out there to try automatically stitching your blob of points into a triangle mesh based on spatial evaluation and assumptions about the geometry's convexity. It's all rather haphazard, but if you spend enough time refining your search/reconstruction criteria, and particularly if you're targeting it to a specific system/architecture, you're likely to come up with a solution that works pretty reliably in automatically finding a good variety of 3D data amongst giant chunks of arbitrary binary. But it's never going to be perfect, and it's going to take a long time to tweak out for relatively little reward in the end, and handling data that tends to vary more (like skeleton structures) is a much bigger problem. Although it still has its own viable methods of approach.

It's a much better idea to simply hook into the rendering API if you can.

MrAdults wrote:Normal maps don't contain a length, they're representing unit vectors, so deriving an actual shape from them would be completely just guess work without height/distance information. A depth buffer gives you a length, but yes, only on the projection axis.

If you had a whole system RAM dump, getting information out of that automatically isn't a 10-years-down-the-line problem. If you make the initial assumption that positions are as 32-bit floats, you can scan the entire memory for valid floating point values. If you find 3 or more in a row, you then scan nearby memory. If you have a giant block of them you may have a vertex buffer, or if you have segments of 3 or more valid floating point values on a fixed stride apart then maybe you've found an interleaved vertex buffer. Then you can try interpreting those values in a variety of ways. You can tally them up and pay attention to whether you've got a flat block of positions or an interleaved buffer to try to determine the total number of vertices, and use that knowledge to even further scan for 16-bit or 32-bit index buffers that fall into a range you expect based on your counts. Or you can rely on one of the many semi-reliable algorithms out there to try automatically stitching your blob of points into a triangle mesh based on spatial evaluation and assumptions about the geometry's convexity. It's all rather haphazard, but if you spend enough time refining your search/reconstruction criteria, and particularly if you're targeting it to a specific system/architecture, you're likely to come up with a solution that works pretty reliably in automatically finding a good variety of 3D data amongst giant chunks of arbitrary binary. But it's never going to be perfect, and it's going to take a long time to tweak out for relatively little reward in the end, and handling data that tends to vary more (like skeleton structures) is a much bigger problem. Although it still has its own viable methods of approach.

It's a much better idea to simply hook into the rendering API if you can.

Well.. yes I suppose you're right. Then again, what you just described is sort of the same process as decoding a format manually anyway. Looking for vertex positions in data patterns, refining search parameters, trying again... Come to think of it, why hasn't anyone written a program to do this and display the result? It wouldn't be much good for final output maybe, but it would be a quicker way to figure out where to look...

Though if I had more skill and a lot more patience, I'd probably start on a predictive algorithm like that as a noesis plugin...

On rendering hooks... wasn't that something you had planned on doing for PS2 models with noesis at one point?

I'd be surprised if there aren't programs out there for that purpose. And yeah, it wouldn't be too hard to do it with a Noesis script as well. Although churning through raw binary and doing tons of validation in Python can get slow fast, so a native plugin would be much better-suited to it.

Nope, never had plans to hook any API's directly for PS2 models. Noesis does have a VIFcode "emulator" in it though, which I use to run through it and pull relevant expanded buffers and such out. It's used in The Bouncer, Bujingai, and maybe 1 or 2 other PS2-based formats I've got in there. Kind of a different concept, because it's implemented on a per-game basis where I know exactly what I'm looking for. Hooking a rendering API generically still involves a bit more guesswork depending on where in the pipeline you're hooking in. (but it depends on the API too, back in the fixed function OpenGL days you could be pretty sure that data being bound for "positions" was actually positions, but shaders and generic buffer binds make intercepting things at that point in the rendering pipeline trickier)