XeNTaX Forum Index
Forum MultiEx Commander Tools Tools Home
It is currently Thu Jun 21, 2018 11:16 am

All times are UTC + 1 hour


Forum rules


Please click here to view the forum rules



Post new topic Reply to topic  [ 102 posts ]  Go to page Previous  1 ... 3, 4, 5, 6, 7  Next
Author Message
 Post subject: Re: Havok binary files
PostPosted: Wed May 18, 2016 3:09 am 
Offline
advanced

Joined: Mon Aug 10, 2015 7:03 am
Posts: 40
Has thanked: 14 times
Have thanks: 5 times

volfin wrote:
HunterAP wrote:
How did you manage to convert from the HKX bone coordinates and rotations to SMD format?
If I could find a way to batch the process, I could get full DSIII animations out.


This is a project I started last year for Alien:Isolation, it's a HKX converter that outputs SMD skeleton. It works with 2012.2.0 and 2010.2.0, I had intended to take it all the way with support for many versions, and animation output, but I simply moved on to other projects. It's written in C#, a lot of the file structure is defined at the top of Program.cs, and maybe will help people, so I'm making the code public.

https://bitbucket.org/Volfin/hkx2smd/overview


I tried building your program as both x86 and x64, normally running the 2014.1.0-r1 HKX files give some errors in the header, but if I copy-paste the 2012.2.0-r1 header over the original, the program reads everything and gets up to "class hkaskeleton signature: 0xFEC1CEDB", then after a few seconds the program crashes with the "System.OutOfMemoryException" error. Looking it up says that I don;t have enough memory, even though I have 16GB of RAM, but I'm assuming that your program is looking for a big chunk of sequential memory that I may not have if my memory usage is fragmented.

What's interesting is that I've tried this with multiple skeletons, and it doesn't matter what size the HKX file is, I still get the error.

I would fix the issue myself, but I'm not versed in C#, so I don't know how to fix it in the code myself.

If you'd like, I could just send you a HKX sample of the skeleton and animations.

You can make the ads go away by registering



Top
 Profile  
 
 Post subject: Re: Havok binary files
PostPosted: Wed May 18, 2016 5:46 am 
Offline
ultra-veteran
ultra-veteran
User avatar

Joined: Sun Jul 06, 2014 6:30 am
Posts: 363
Has thanked: 85 times
Have thanks: 243 times
Well the point of me releasing it is that I have no time to mess with it anymore (haven't touched it since Alien:Isolation). I know it worked for Alien: Isolation and Skyrim, but don't guarantee anything else. The problem is HKX is very customizable so the format varies widely from game to game, and then on top of that many developers modify it in-house to do even more strange things, and some even obfuscate it purposely to prevent people reverse engineering their animations.

So likely the code will have to be tweaked for every particular game, no small job. That's why I gave the code out. :)


Top
 Profile  
 
 Post subject: Re: Havok binary files
PostPosted: Mon Mar 06, 2017 6:52 pm 
Offline
ultra-n00b

Joined: Mon Mar 06, 2017 6:11 pm
Posts: 5
Has thanked: 0 time
Have thanks: 3 times
As I understand, the spline compression is a technic that is animation specific.

See this article at riot game engineering
https://engineering.riotgames.com/news/compressing-skeletal-animation-data
So, it is likely there is no simple link between the hkx file and the animation keys.

Maybe the way hkx animation is encoded is similar to the one described in the article that is to say :
- rotation is encoded as quaternion (x,y,z,w),
- the highest value among (x,y,z,w) is discarded as x^2+y^2+z^2+w^2=1 and can thus be reconstructed from the 3 others,
- bytes need to be read according to a key (m bytes for discarded value, n bytes for remaining value1, n bytes for remaining value2, n bytes for remaining value3)

edit : this article at bitsquid describes also the technic
http://bitsquid.blogspot.fr/2009_11_01_archive.html

Best regards

Phil


Top
 Profile  
 
 Post subject: Re: Havok binary files
PostPosted: Fri Mar 10, 2017 6:52 pm 
Offline
mega-veteran
mega-veteran

Joined: Fri Aug 30, 2013 5:39 am
Posts: 236
Has thanked: 55 times
Have thanks: 19 times
The SDK also has a LOT of documentation files in it, including files for all the compression types instead of just Spline. It's not available through Intel anymore but I managed to save copies. Feel free to PM me for the link.


Top
 Profile  
 
 Post subject: Re: Havok binary files
PostPosted: Sun Mar 12, 2017 11:26 am 
Offline
ultra-n00b

Joined: Mon Mar 06, 2017 6:11 pm
Posts: 5
Has thanked: 0 time
Have thanks: 3 times
Sergeant Joe,

I thank you for your reminding of the Intel documentation.
I have downloaded havok 2013 before Intel closed the public access.
At the time I downloaded, I couldn't make any sense of spline compression.

Now, I understand better :
After spending some time on xentax forum, I started to get acquainted with binary files ans lately with file compression.
Following your advice, I looked again at havok 2013 help files.

The game I'm interested in is using spline compression.
So, I restricted my field of research the spline compression.

There are havok default settings for spline compression :
Position : 16 bits
Rotation : 40 bits
Scale : 16 bits
Float (what for ? timeline ?) : 16 bits
Frames per block : 256

As the spline compression is highly customizable(*) in havok, this should only be considered as a reference.
At least, it gives the structure of the data.

I am a beginner at reading binary files but it looks like an interesting challenge :

- identify blocks,
- identify number of frame per blocks,
- guess bone ordering,
- guess quantization for each bone,
- translate into skeleton pose,
- translate into animation.

I need to evaluate the required time.

Best regards

Phil

*() : Even, a per bone customization is possible. The doc says :
"Judiciously setting compression parameters per bone can result in a significant reduction in output data size. One such use case is to create level of detail compression by grouping bones roughly by depth in the hierarchy; tight tolerances could be user for "major bones" such as spine and arm bones while more relaxed tolerances for "minor bones" such as hands and fingers could be used to gain additional savings. A similar use case might involve grouping bones as "upper body" and "lower body". Extremities such as hands often contain a significant number of bones though their visual impact may be limited."


Top
 Profile  
 
 Post subject: Re: Havok binary files
PostPosted: Mon Apr 03, 2017 7:40 pm 
Offline
n00b

Joined: Thu Jan 15, 2015 10:10 pm
Posts: 15
Has thanked: 1 time
Have thanks: 0 time
Does anyone have a working link for the script that flips the skeleton or model because when I open animation in 3dsmax the skeleton has the right animation but the model is all over the place.


Top
 Profile  
 
 Post subject: Re: Havok binary files
PostPosted: Mon Jul 31, 2017 11:52 pm 
Offline
ultra-n00b

Joined: Fri Mar 24, 2017 12:18 am
Posts: 7
Has thanked: 0 time
Have thanks: 0 time
Would anyone mind helping me out with obtaining the animation data for Dark Souls III?

Regards


Top
 Profile  
 
 Post subject: Re: Havok binary files
PostPosted: Tue Aug 01, 2017 12:49 pm 
Offline
mega-veteran
mega-veteran

Joined: Fri Aug 30, 2013 5:39 am
Posts: 236
Has thanked: 55 times
Have thanks: 19 times
zweihard wrote:
Would anyone mind helping me out with obtaining the animation data for Dark Souls III?

Sorry, you're out of luck. Havok formats are incredibly difficult, even with all the documentation nobody has been able to figure it out.


Top
 Profile  
 
 Post subject: Re: Havok binary files
PostPosted: Tue Aug 01, 2017 9:23 pm 
Offline
ultra-n00b

Joined: Fri Mar 24, 2017 12:18 am
Posts: 7
Has thanked: 0 time
Have thanks: 0 time
SergeantJoe wrote:
zweihard wrote:
Would anyone mind helping me out with obtaining the animation data for Dark Souls III?

Sorry, you're out of luck. Havok formats are incredibly difficult, even with all the documentation nobody has been able to figure it out.


Reverse engineering job huh?


Top
 Profile  
 
 Post subject: Re: Havok binary files
PostPosted: Fri Nov 24, 2017 10:25 pm 
Offline
n00b

Joined: Thu Nov 23, 2017 10:46 am
Posts: 13
Has thanked: 2 times
Have thanks: 0 time
Sorry for bringing up this topic again, but Im kinda stuck with a havok file here.

I got a havok binary with the hk_2012.2.0-r1 header. However, none of the converters seem to work with it (even when renaming the header) and the original havok preview tool says something like "wrong platform".

Anyone has an idea of what is so special/wrong about that file?

hk_2012.2.0-r1 binary


Top
 Profile  
 
 Post subject: Re: Havok binary files
PostPosted: Tue Nov 28, 2017 12:01 pm 
Offline
mega-veteran
mega-veteran

Joined: Fri Aug 30, 2013 5:39 am
Posts: 236
Has thanked: 55 times
Have thanks: 19 times
I honestly have no idea, and there's nobody else that can help. Sorry, you just have to give up.


Top
 Profile  
 
 Post subject: Re: Havok binary files
PostPosted: Tue Nov 28, 2017 12:15 pm 
Offline
n00b

Joined: Thu Nov 23, 2017 10:46 am
Posts: 13
Has thanked: 2 times
Have thanks: 0 time
SergeantJoe wrote:
I honestly have no idea, and there's nobody else that can help. Sorry, you just have to give up.



well, there are some people that made the converters, so there should be some other help out there...in theory haha
however, thanks for looking into it!


Top
 Profile  
 
 Post subject: Re: Havok binary files
PostPosted: Sat Dec 09, 2017 4:51 pm 
Offline
beginner
User avatar

Joined: Sat Sep 12, 2015 2:16 am
Posts: 37
Location: South Africa
Has thanked: 19 times
Have thanks: 6 times
Ginsor wrote:
Sorry for bringing up this topic again, but Im kinda stuck with a havok file here.

I got a havok binary with the hk_2012.2.0-r1 header. However, none of the converters seem to work with it (even when renaming the header) and the original havok preview tool says something like "wrong platform".

Anyone has an idea of what is so special/wrong about that file?

hk_2012.2.0-r1 binary



Been a while since i tried my luck figuring this out, but just if it helps: look out for the c# project shared by the awesome (if not mistaken) SeargentJoe, (hkx2smd, and there is another one, or similar project on github, with some fiddling managed to parse some basic 2013 structs, im looking to research splinecompressed anims & bone/weight/blendweight conversion. though quite difficult to find older/recent havok src. (looking for 2011r3 & 2013+)

Just off top of my head, and old snippet idk if works:- look for this flag in ur data which specifies platform type and modify to ur build platform. also thinking might be some endianness conversion needed ( ? )

public enum hkPlatform
{
HCL_PLATFORM_WIN32 = 0x0,
HCL_PLATFORM_X64 = 0x1,
HCL_PLATFORM_MACPPC = 0x2,
HCL_PLATFORM_IOS = 0x4,
HCL_PLATFORM_MAC386 = 0x8,
HCL_PLATFORM_PS3 = 0x10,
HCL_PLATFORM_XBOX360 = 0x20,
HCL_PLATFORM_WII = 0x40,
HCL_PLATFORM_LRB = 0x80,
HCL_PLATFORM_LINUX = 0x100,
HCL_PLATFORM_NGP = 0x400,
HCL_PLATFORM_ANDROID = 0x800,
HCL_PLATFORM_CTR = 0x1000//4096
}
if u look up my old posts, io posted 010 template for havok while back, somewhere on forum, including offset of Platform flag (i didnt have much joy parsing PS3 files on win32 build)


Top
 Profile  
 
 Post subject: Re: Havok binary files
PostPosted: Sun Dec 10, 2017 3:19 am 
Offline
advanced

Joined: Mon Aug 10, 2015 7:03 am
Posts: 40
Has thanked: 14 times
Have thanks: 5 times
After all this time I'm also trying to get some newer formats of HKX working. Right now, Volfin's hkt2smd is crashing because newer formats report that for a variable used in the code, there is over 1.5 billion elements in an array (AKA way too much to work with), and I'm not sure how to go about patching it.

Also, interestingly, trying to use hkt2smd on Dark Souls 1 has a similar error but in a different section.


Top
 Profile  
 
 Post subject: Re: Havok binary files
PostPosted: Sat Dec 16, 2017 8:37 am 
Offline
beginner
User avatar

Joined: Sat Sep 12, 2015 2:16 am
Posts: 37
Location: South Africa
Has thanked: 19 times
Have thanks: 6 times
HunterAP wrote:
After all this time I'm also trying to get some newer formats of HKX working. Right now, Volfin's hkt2smd is crashing because newer formats report that for a variable used in the code, there is over 1.5 billion elements in an array (AKA way too much to work with), and I'm not sure how to go about patching it.

Also, interestingly, trying to use hkt2smd on Dark Souls 1 has a similar error but in a different section.


can u post here sample of both assets ur trying to parse ? i can take a look.



Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 102 posts ]  Go to page Previous  1 ... 3, 4, 5, 6, 7  Next

All times are UTC + 1 hour


Who is online

Users browsing this forum: daemon1 and 14 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group