Metal Gear Solid 5 (MGSV) animations

[MrAdults]

Since we're all doomed to eternal darkness anyway, those wasted hours I could have used to get drunk and be merry.

[daemon1]

How can you say this, when you've just spent weeks on an animation format

A week is not that much time. I usually only have a couple of hours in the evening for research, and not every day. So if we count the actual time spent it will be about 10 hours for a normal game. This including research of a model format, because it will be needed for animation, writing a tool, correcting all bugs, exporting some models and testing how it all work together. Do you say it all can be done in 1-2 hours?

You know as well as I do, you will make guesses, and they will be wrong.

No I will not. This may sound strange, but it's true. In most cases the format can be solved without wrong guesses. If this happens, I call it a flawless solve.

For example we have a model file I recently did - Hellboy Science of Evil PS3. In the beginning I can see a list of some offsets. This will probably be offsets to file segments with different model data, such as bones, geometry, etc. And this guess was right. I'm checking them. Some of them I can't identify yet, but this is not a wrong guess. I'm just leaving them for later.

Now one of the data arrays definitely look like bone matrices. They are easy to detect. I'm checking them and counting them. Looking at the float numbers, I detect where the global bone coordinates are. Later we'll know this was also right guess. Now I have bone count and coordinates. Checking this count and I found it in both model and animation file. This means now I know where it is written in the header. Another right guess.

Now when I know the total bone count it looks like another array I've seen before is bone remapping, because it consist of a number of lists having all numbers from 0 to total bone count minus 1. And later I'll see that this was again the right guess.

This way, one thing after another, I can solve the whole format. Of course, there may be some parts unknown for me left, but this will only happen in case I don't need them. But as soon as I'll realize I need something else, these will be easy to identify, because with each step, the amount of unknown data decreases. Just like in solving a nonogram.

So as you can see, no time wasted, everything is solved, no mistakes, no wrong guesses.

I got my first gamedev contract about 20 years ago

Well, I didn't say I was WORKING in this for 30 years. Most of the time it was for fun. And I never worked in gamedev. Actually I returned to this game-analyzing stuff only recently, about 3 years ago, and it was like a fresh air to me. I actually discovered what quaternions are only this spring. The time for learning what they are is also included in those 10 hours mentioned before. And I can't say they are something intuitively understandable

Also I never said I don't use disassembly. I actually said I never decompile the whole anim engine to put it into my tool. Once I've seen that quats in morpheme are packed some unusual way, I ran the debugger, found a place where it's done, and very soon I knew it and returned back to analyze.

The book is very well organized

Maybe you could say that 5 or 10 years ago. But now with all this multi-threading, FPU and SSE mixed up in one line for optimization, I can actually SEE when looking at the data and the code that reads this very data, how much easier it is to understand it by data, rather then code. I've seen how they read sets of ints for 4 bones from memory, converting them to float and making SSE calculations all messed up one with another. No matter if you look at assembly or decompiled c++ it's hard to understand what it does. I think you saw it too.

The simple xor-decrypt loop optimized for 8 threads looks like a monster, and I don't know of a tool that is able to decompile it into normal human-readable code.

You're advocating a methodology that will have that very result for beginners because they'll just stare at and "analyze" data as you're advocating, until they hit their fateful wall, and retreat back into the sandbox.

If they take your advice and try assembly only, there's a big chance they will find it too difficult and give up for good. Again, I'm not against disassembly, I was just disagree that only this method should be used always as it seemed to me from your previous quotes. But now it looks like you don't actually mean that:

In practice, I rarely need to take the debugging & disassembly route

So now I think we can summarize that beginners should learn both methods and use what suits them most.

[MrAdults]

A week is not that much time. I usually only have a couple of hours in the evening for research, and not every day. So if we count the actual time spent it will be about 10 hours for a normal game. This including research of a model format, because it will be needed for animation, writing a tool, correcting all bugs, exporting some models and testing how it all work together. Do you say it all can be done in 1-2 hours?

When you account for normal human scheduling and needs, for me, an animation format like this that has a lot of nasty variations on packing that utilizes multiple spaces (which typically will only happen if IK is employed) should take something in the area of 10-20 hours with effective use of disassembly. Unfortunately, I rarely get that much time to spend in a month myself. Something simple like the Space Channel 5 animations I looked at the other night took about an hour from start to finish, including writing up the new Noesis import logic to handle it, tying it to the models, and remembering how the Ninja model format was set up. So it's very subjective. Models will generally take no time at all unless there is something very new and crazy, like the first time I ran into Edge index compression. But this has nothing to do with disassembly being inefficient, it's just because my knowledge/intuition has become vast after years of working with this data. That will happen whether you've been employing disassembly or not. This intuition in many cases is built upon prior use of disassembly to get me over a hurdle more effectively.

No I will not. This may sound strange, but it's true. In most cases the format can be solved without wrong guesses. If this happens, I call it a flawless solve.

Sure, if your data is perfectly intuitive (like so many model formats), you're not going to waste time trying things that don't work. Your example is many worlds apart from a format that uses a proprietary means of tightly packing data through, for example, some form of variable delta compression. I'm not advocating people use disassembly to solve things that are stupidly obvious, but I'm advocating that they be able to do so if they need to. If you can't understand a quaternion or a unit-length vector when you spot it in hex, you have a lot of growing to do before any approach will be effective for you in the realm of models/animation. But you probably understand these things already if you've figured out just about any modern game + skeleton through analysis, as most people on these forums already have. Quaternions are extremely common, although I'll give you that inverse matrix transforms tend to be more popular for representing the base skeleton, for the purpose of transforming modelspace transforms for the bind pose. (with the exception of those rare games that still choose to store vertices in local space)

I actually discovered what quaternions are only this spring. The time for learning what they are is also included in those 10 hours mentioned before. And I can't say they are something intuitively understandable

Once you understand that a quaternion is just a rotation about an axis, it's pretty intuitive. But a lot of people can never seem to wrap their brain around it. This is why I included "sometimes it's best to broaden your knowledge in the field instead of continuing to reverse engineer data you don't understand" in my initial list of suggestions.

Again, I'm not against disassembly, I was just disagree that only this method should be used always as it seemed to me from your previous quotes.

Yeah, it's silly to use disassembly when you can look at something for an hour and have it all mapped out. Your methodology of mapping a file out I think is very common between beginners of veterans alike, the problem is when you come upon some part of that data that you spend too many hours trying to understand and there is nothing to "eliminate" to make it easier. That's when it's smarter to use disassembly. If you say you immediately intuited all of the packing types used for MGS5, I won't believe you. But if you can't use disassembly when you hit a wall, then you're just stuck. If you haven't been using it, it will take you hours or days (and I don't mean 1-2 hour days) of time to figure out how to get where you need to be. This will discourage you from switching to it, meaning you'll just keep hitting your head on the wall. Maybe eventually you'll get lucky, but you'll have wasted more time than if you had the confidence to go right over to disassembly.

Also I never said I don't use disassembly. I actually said I never decompile the whole anim engine to put it into my tool. Once I've seen that quats in morpheme are packed some unusual way, I ran the debugger, found a place where it's done, and very soon I knew it and returned back to analyze.

Well, where you cut the line is up to you, but it's very effective either way. For example, if you have many pages of disassembly with a targeted purpose, like decoding tightly packed data with many different conditional branches, there is no reason not to isolate that code. Not to mention, if you need to go back to it to validate results, you already have it right there in front of you. No need to fire up the game and debugger all over again.

Maybe you could say that 5 or 10 years ago. But now with all this multi-threading, FPU and SSE mixed up in one line for optimization, I can actually SEE when looking at the data and the code that reads this very data, how much easier it is to understand it by data, rather then code. I've seen how they read sets of ints for 4 bones from memory, converting them to float and making SSE calculations all messed up one with another. No matter if you look at assembly or decompiled c++ it's hard to understand what it does. I think you saw it too.

The simple xor-decrypt loop optimized for 8 threads looks like a monster, and I don't know of a tool that is able to decompile it into normal human-readable code.

You can still say it. That is why "practice makes something resembling perfect". Compiler optimizations (particularly with SSE, I'll agree to that) and the types of job systems developers like to employ these days make it a greater kind of hell, but that's another thing where understanding what to look for, your "disassembly intuition", is very helpful. You can see where a kernel is being pushed to a job system, for example, and that actually makes your life even easier. You have that kernel right there nice and isolated to show you exactly how your data is being processed, and better yet, you probably have a structure that kernel is using that points to all the data you care about. With so many developers switching to fiber-based job systems, or otherwise a system that simply pushes some job structure to a queue, you just have to understand how that system is working. Knowing how to recognize that you're in a kernel/fiber in a lot of ways makes the job even easier and it makes it easier to isolate parts of the system.

And yeah, there is a learning curve to understanding code with compiler-generated SSE/SSE2/NEON/etc. in the mix. But you get used to it, especially if you've spent a lot of time debugging problems in this same optimized disassembly like me. You start to develop a greater intuition of what that optimized code came from or the kinds of initially-confusing decisions the compiler makes to optimize for SIMD or utilizing separate SIMD registers.

So now I think we can summarize that beginners should learn both methods and use what suits them most.

I can generally agree on that.

I'm pretty sure the vast majority of people doing work here on Xentax, especially with 3D models and animation, have little to no disassembly experience. Hence the emphasis on learning this upfront as a tool to overcome imminent hurdles. Disassembly is a different discipline with a different type of intuition associated with it, and it will help you along the way. Of course, there are also ways to help that disassembly intuition that doesn't involve just disassembling in itself. I got a lot of free practice from debugging "impossible" crashes in multi-threaded scenarios with optimized and sometimes even intermingling of managed code in the mix. Maybe "never give up on a crash" is also good advice to accidentally becoming better at parsing disassembly.

Our workflows are probably very similar, in the end, except I view switching over to disassembly as a time-saving measure instead of a disappointment.

[daemon1]

utilizes multiple spaces (which typically will only happen if IK is employed)

Do you mean local/global space or something else? I may be not too good in terms, I'm no animator How would you do exporting in this case, I mean MGSV? Does noesis have some kind of IK-solver, or you would try to isolate their IK-solving code and directly putting it into your program? How many games have you seen utilizing this kind of FK/IK mix, because another professional animator told me this is really exotic.

Your example is many worlds apart from a format that uses a proprietary means of tightly packing data

Yes. But still, I think 90% games are not using any tight packing. Packed quats and half-floats best case. Nothing more complex.

For example, if you have many pages of disassembly with a targeted purpose, like decoding tightly packed data with many different conditional branches, there is no reason not to isolate that code

I actually saw how these "many pages of disassembly for tightly packed data" can be replaced with only a few lines in C after a quick research in debugger. From what I've seen, "code isolation" is only needed in case of complex math, like audio codecs. Yes, I've seen not too many cases, and maybe there are harder ones, like maybe MGSV, but this is probably very rare.

[MrAdults]

Yeah, I was referring to local and model space. It's pretty rare these days to see animations stored completely in model space, although frequently certain hierarchies are "flattened" so they're effectively in model space in order to reduce transform overhead. Most commonly, everything is in local space. But if you're going to animate IK effectors, it makes sense to do it in model space, to make it easy to modify those effectors as-needed at runtime.

Animating IK joints and/or effectors is something I hadn't seen much of since the PS2 days. I have a 2-bone and an analytic IK solver buried in the noesis_misc module, but haven't exposed it in the public API, as animation interest hasn't been too high until, strangely, very recently. The problem is that games like to store relevant data in very different ways, especially constraints. I don't know why it was so popular specifically on PS2 titles, I think this is either a Japanese developer thing or perhaps Sony provided some common VU code/libs that hinted in this direction.

I've never seen a reason to approach IK in that way, it generally makes more sense to perform FK animation transforms, take those modelspace transforms and use them to apply IK with constraints, and optionally blend the IK in on a given hierarchy. This is the way I've always implemented it myself, and have no plans to switch over. Animating IK effectors is also basically ensuring you always have to solve for IK on a given hierarchy, which is kind of a pointless overhead, but it makes ok sense to have it only done on a per-sequence basis. (and the plus side is that this gives the animator better visibility as they're creating the animation, instead of only having the IK solver applied at runtime, and messing their animation up in a way they didn't realize would happen)

How to preserve the data is up to you - it mostly depends on how complicated it is. If you need to preserve some detailed constraint information, you might be best off dumping it out in a paired file, or a .qc file has plenty of support for constraints if you can make the data conform, since you're already exporting to .smd from the looks of it. Then you just dump the anims as FK transforms. When you export .smd back to MGS5 anims, pull the same constraint data back in (probably based on joint names), and take the modelspace transforms of the effector joints to automatically convert them back into IK sequences where needed. (you'll probably need some data preserved to also determine which animations need which joints/hierarchies IK-driven, you might just adopt a joint naming convention to make this easier) Depending on how the IK joints are animated, those might restrict the allowable techniques animators can use to change those animations (like a lot of PS2 games will animate nothing but a simple effector position for some joints, making it impossible to control custom rotations outside of the solver), but that's Just Life.

[daemon1]

Animating IK joints and/or effectors is something I hadn't seen much of since the PS2 days

Thanks for all the info. If even you say it, then what they did is really rare.

If you need to preserve some detailed constraint information

I'd be happy if I can do it without constraints, but now I don't see any other way to export data.

Then you just dump the anims as FK transforms.

Unfortunately I can't do this. There are just no FK transforms for IK bones in game files.

you'll probably need some data preserved to also determine which animations need which joints/hierarchies IK-driven, you might just adopt a joint naming convention to make this easier

I can't do this either. This is already defined in game for the whole character model, not individual animations.

So this game is probably the hardest game in the whole world to export animations.

[MrAdults]

Unfortunately I can't do this. There are just no FK transforms for IK bones in game files.

I meant, when you want to convert back into the game format from a limited FK-only format, you can convert your FK transforms (that you've generated by solving the IK transforms and exporting to SMD as FK) back into IK, by transforming your FK joints to modelspace, and deriving the IK transform from the modelspace transform for the joint.

I can't do this either. This is already defined in game for the whole character model, not individual animations.

So this game is probably the hardest game in the whole world to export animations.

At least, in a way, that makes it easier, since you know something is or isn't an IK joint globally. That's surprising though, that they would just have IK always active for all sequences.

But yeah, sounds like a pain in the ass format.

[daemon1]

At least, in a way, that makes it easier, since you know something is or isn't an IK joint globally. That's surprising though, that they would just have IK always active for all sequences.

Yes, another strange japanese thing. And they have different IK node types hardcoded in their engine. I mean they have different sets of parameters, and processed differently. I already exported human leg and human arm types, now working on robot leg. And there is also some other type in horses. Also there are types I can't find used anywhere yet, but maybe they are for some weird types of monsters/animals.

[daemon1]

New cutscene tool. Phantom Pain cutscene support! All characters (including dogs, horses, etc)

[daemon1]

Update. Forgot to remove some debug constants. Should be fine now.

[rubening]

i cannot believe you did this omg THIS WAS SO NICE!. really nice!!

One question regarding other games. we have tools / script for importing exporting meshes + skels in resident evil 5 and 6, do you think that would help in the process to get the animations from the games?

Animations from re5 and re6 (specially melees and biped interactions) are very cool in those games.

i wouldnt mind to donate if you could do this.

[daemon1]

do you think that would help in the process to get the animations from the games?

Models are needed to get animations, yes. Main question is what type of anim engine they use, some third party, or their own.

[rubening]

do you think that would help in the process to get the animations from the games?

Models are needed to get animations, yes. Main question is what type of anim engine they use, some third party, or their own.

to be honest i think their own, but i'm not sure. I have Lmt files(re5format animation files) from the game and i've the script to export the models too.

probably GhostBlade knows about this answer...would be nice if he could answer this question he knows a lot about this games file format.

probably the animations are the same format in devil may cry 4... Because at least the models are on the same format.

by the way could i get the.. snakes running animation without downloading the game? if you could share with me. That animation looks really good.

[tpcrew]

New cutscene tool. Phantom Pain cutscene support! All characters (including dogs, horses, etc)

Hi,
this is great! But I have a question, is there a tool to extract and import the models in 3dsmax?
thanks.

ps. I apologize if my question does not go in this thread.

[daemon1]

probably the animations are the same format in devil may cry 4... Because at least the models are on the same format.

I checked animations, and they are not too complex. I can even export them, but the problem is, the bone numbers don't correspond to the bones in model. So I need information why model is exported that way. I can't apply animation to bones without it.

You said its like devil may cry 4? devil may cry 4 anims can be exported?

by the way could i get the.. snakes running animation without downloading the game? if you could share with me. That animation looks really good.

No, because I don't know where is running animation, there are about 2000 of them there.