XeNTaX

Posted: **Tue Nov 08, 2016 7:11 pm**

unknown's writeup on importing appeal actions to single player inspired me to look into the .mtar format. I've still got a lot of progress to make, but this is my progress so far:

Code: Select all

Mtar Info:
Uses Little-Endian
0x0 - 0x3 (32): Signature? 9C A8 FF 0B = Gzs, 72 2B 01 0C = Tpp
0x4 - 0x7 (32): Number of files. Each line (or 2 lines[?] for player2_resident.mtar?) starting at 0x20 seems to indicate a file
0x8 - 0xB (32): Unknown. TppGzPlayerLayers and player2_resident share bytes 12 00 38 00. TppGzPlayerFacial (26 00 44 00) and TppPlayer2Facial (23 00 44 00) slightly differ.
0xC - 0xF (32): Unknown. TppGzPlayerLayers and player2_resident share bytes 00 00 00 00. TppGzPlayerFacial and TppPlayer2Facial share bytes 06 00 01 00.
0x10 - 0x1F: Unknown.

File declaring/listing begins at 0x20. All files in the list follow this format:
(Note: some files i.e. player2_resident seem to use a different format)
0x20 - 0x27: Unknown. Best guess is this might be the hashed name of the file. I.e. 63 0C 20 DA FA AF 53 = 3affada200c63
0x28 - 0x2B (32): File location offset.
0x2C - 0x2F (32): File length.

Gani File format:
0x0 - 0x4 (40): Signature (D2 A2 FC 0B 20).
0x5 - 0x7 (24): Padding.
0x8 - 0xB (32): File length.
0xC - 0xF (32): Padding.

From here it gets more confusing. What I assume are .gani files in some of the files, seem to be lacking a distinguishable header. The offset for file length also doesn't seem to match up. There also seems to be an additional line attached to each file in the file list. I haven't figured out what this additional line does or means yet.

Edit: I can confirm that my .gani format notes are correct now. I found some .gani files sitting outside of a .mtar in one of GZs .fpks. Unfortunately, this doesn't help much with the second .mtar format. Still trying to figure that one out.

Posted: **Tue Nov 08, 2016 7:46 pm**

BobDoleOwndU wrote:Mtar

Just a wild guess, but can it be based on .tar format? Since fmdl is 'Fox mdl', then mtar could be 'Motion tar' as well.

Fv2 progress:
I think I am ready to make a lot of fv2 files for headgear. Haven't touched body gear yet.
Issues:

head models for snake and dd soldier (male) use one fova, but their main models are just a bit different.

See https://imgur.com/a/kE23b - soldier has his glasses a little bit farther from where it should be, while snake looks like that guy from MiB3. I think all head cosmetics will suffer from that issue in one way or another as long as they use the same model.

You will have to swap fv2 files inside fpks and repack them, no quick change allowed due to the issues related to ApplyFormVariationWithFile stated in a couple of posts back.

--------------------
or not.. For some insane reason mgo models doesn't show if they are attached using method I was using before. What a mess.

Posted: **Tue Nov 08, 2016 8:45 pm**

Code to S rank and complete all mission objectives. Call from end of TppMain.OnInitialize func:

Code: Select all

function this.CompleteAllMissionTasks()

	local missionsToTinker = TppDefine.MISSION_ENUM
	local setScore=200000 --preferably over 120,000
	local setRank=1 --1=S, 2=A, 3=B, 4=C, 5=D, 6=E

	for missionCodeString,enumId in pairs(missionsToTinker)do
    local missingMissionEnumId=TppDefine.MISSING_NUMBER_MISSION_ENUM[missionCodeString]
    if not missingMissionEnumId then
    
      for i=0,TppUI.GetMaxMissionTask(tonumber(missionCodeString))-1 do
		    local n=enumId*TppDefine.MAX_MISSION_TASK_COUNT+i
		    gvars.ui_isTaskLastComleted[n]=true
		  end
		  
		  gvars.rnk_missionBestScore[enumId]=setScore
		  gvars.res_bestRank[enumId]=setRank
		  TppStory.SetMissionOpenFlag( tonumber(missionCodeString), true )
	    TppStory.UpdateMissionCleardFlag(tonumber(missionCodeString))
	    --TppStory.UpdateStorySequenceOnMissionClear(tonumber(missionCodeString)) --Uncomment to progress story

    end
  end
end

Need to complete at least one mission to unlock more stuff like ninja outfits

Posted: **Wed Nov 09, 2016 1:32 am**

unknown123 wrote:Just a wild guess, but can it be based on .tar format? Since fmdl is 'Fox mdl', then mtar could be 'Motion tar' as well.

I took a brief look at the .tar format, but didn't notice any obvious correlation. I haven't ruled it out yet though. I'll need to take a more in-depth look at the format to be sure.

It would make sense if it is based off of another format though considering it doesn't have seem to have a similar signature to other Fox Engine archive formats. I.e. QAR files have "SQAR" as their first 4 bytes in plain text, FPKs have "foxfpk" FPKDs have "foxfpkd", etc....

Posted: **Wed Nov 09, 2016 4:58 am**

nasanhak wrote:Here's the summary if you don't feel like going through the sheets.

Sweet, I was wondering what all of those were for. I tried looking for patterns awhile back and ran into a lot of scenarios like: "Oh p##, it's obviously a parameter only for items with... a battery/timer? I guess that makes sense. Is that really it? I mean there's one item that should fall under this that doesn't, but I might be close. Nope there's a damn theory-destroying cardboard box hiding in my search."

QAR dictionary project fork. Fork of a fork of a fork where I'm uploading updates to the dictionary.

all unique qar filepath hashes

I modified my lua script that brute-forces lng dictionary entries to generate QAR entries with unknown123's mgsv_path_hasher. I initially gave up on the idea of integrating it but when I checked the page for it again I noticed that the reason the results didn't match the filepath hashes was that you have to run it from cmd with the /r switch to keep it from reversing hashes. TPP's filepath hashes are 8 - 13 lowercase hex characters, with the 13 character hashes using numbers 1-3 for the leading character. mgsv_path_hasher generates 12 character strings which match up with the filepaths; apparently KJP just set up the filepath hashes to randomly be truncated to 8 - 12 characters and/or generated with 13 characters with a pseudo-random/random character in the range of 1-3. I coded around it by just using 11 - 13 character length hashes that I truncated to the latter 11 characters (wth is the term for the ending character of a string?), and stripping the leading character from mgsv_path_hasher's generated hash with pattern matching when importing output.txt into a table. I haven't seen any collisions so far in the dictionary's appended entries but bringing the hashes down to 8 characters to try get everything would probably introduce issues since the script writes up to 1.4m (around a 100 mb .txt file for paths) in 20+ seconds at a time, so a collision would still be unlikely unless my napkin math is off, but I'd need to verify that before changing the hash table compares around.

Anyway here are some stats and patterns with the hashes for anyone that can make use of it:

Code: Select all

--unique hashes in TPP game files

--[[
|Totals|
all: 44,024
8 char: 1
9 char: 2
10 char: 37
11 char: 650
12 char: 10,220
13 char: 33,114

collisions (trunc. 8 char): 0

|Frequency of leading char in 13 char hash|
0: 0
1: 11,108
2: 10,959
3: 11,047
4-9, a-f: 0
]]

I'll edit in the script in whenever I clean up some of the unused and temp debug code.

Posted: **Wed Nov 09, 2016 9:18 am**

unknown123 wrote:Just a wild guess, but can it be based on .tar format? Since fmdl is 'Fox mdl', then mtar could be 'Motion tar' as well.

I believe it's just short for motion archive.
Similarly .qar comes from the .dar (data archive) files from MGS1 but they seem to adjust the format with each game despite reusing the extension names.

Posted: **Thu Nov 10, 2016 12:01 pm**

Hi all, it's been a while.

@tex, I was hoping you could implement a "skip the splashscreen logos" toggle to IH. I wanted to make a standalone mod for it, but it edits init_sequence.lua in the init.fpkd, which is also used by IH.

The change would simply be:

Code: Select all

function this.StartPreTitleSequence()
  local slot = TppDefine.SAVE_FILE_INFO[TppScriptVars.CATEGORY_GAME_GLOBAL].slot
  TppScriptVars.CopySlot( slot, { TppDefine.SAVE_SLOT.SAVING, slot } )
  TppScriptVars.LoadVarsFromSlot( TppDefine.SAVE_SLOT.GLOBAL, TppDefine.VARS_GROUP_GAME_DATA_ON_START_MISSION, TppScriptVars.CATEGORY_GAME_GLOBAL )
	SplashScreen.Delete(SplashScreen.GetSplashScreenWithName("konamiLogo"))
	SplashScreen.Delete(SplashScreen.GetSplashScreenWithName("kjpLogo"))
	SplashScreen.Delete(SplashScreen.GetSplashScreenWithName("foxLogo"))
    this._StartPreTitleSequence()
end

Instead of:

Code: Select all

function this.StartPreTitleSequence()
  local slot = TppDefine.SAVE_FILE_INFO[TppScriptVars.CATEGORY_GAME_GLOBAL].slot
  TppScriptVars.CopySlot( slot, { TppDefine.SAVE_SLOT.SAVING, slot } )
  TppScriptVars.LoadVarsFromSlot( TppDefine.SAVE_SLOT.GLOBAL, TppDefine.VARS_GROUP_GAME_DATA_ON_START_MISSION, TppScriptVars.CATEGORY_GAME_GLOBAL )

  if not TppSave.IsNewGame() then
    TppSequence.SetNextSequence("Seq_Demo_ShowKonamiAndFoxLogo")
  else
		
		SplashScreen.Delete(SplashScreen.GetSplashScreenWithName("konamiLogo"))
		SplashScreen.Delete(SplashScreen.GetSplashScreenWithName("kjpLogo"))
		SplashScreen.Delete(SplashScreen.GetSplashScreenWithName("foxLogo"))
    this._StartPreTitleSequence()
  end
end

What do you think?

Posted: **Sat Nov 12, 2016 9:02 pm**

Hey all, glad to see the MGSV modding community is still around. Don't suppose anyone has found something that would explain how the game decides which models don't show up in first person when using the int scope?

Almost seems like it would be something contained within the .fmdl files that the game looks for, or maybe it could be related to .fv2 files somehow? Of course can use the arm slotting tricks to get around it but being able to prevent any model from showing up in first person would open up a lot of more possibilities with model swapping.

Posted: **Sat Nov 12, 2016 9:12 pm**

TruckerHatRyan wrote:Don't suppose anyone has found something that would explain how the game decides which models don't show up in first person when using the int scope?

You mean parts of player's model? I think it is hardcoded and depends on camera. You see, there is a bug, which allows you to see through intscope, but in third-person mode. Your body is fully seen along with your head. So it depends on camera distance. I haven't seen anything camera-related in fv2 yet.
http://i.imgur.com/RKyVTfN.jpg - pic of the bug, I cannot reproduce it reliably.

Posted: **Sat Nov 12, 2016 9:26 pm**

unknown123 wrote:You mean parts of player's model? I think it is hardcoded and depends on camera.

Yep that is what I was referring to, how on models that were not intended to be playable you will see them in first person when using the int scope if you make them the main .fmdl of an fpk.

That is unfortunate if it is hardcoded, but I suppose that does make sense. I seem to recall someone finding some camera related lua files and I think Shigu (or Higus on this forum) was looking into using those to solve the same problem but looks like it may have been a dead end.

Posted: **Sat Nov 12, 2016 9:59 pm**

@morbidslinky
I had a logo skip back in version r41 but removed it in r90 (well actually replaced) because:

It was before I knew how to pull pre-load save values so I couldn't have it as a toggle (no longer an issue).

The only situation where logos would increase the load>title screen time is if the load time is less than the logos appear time.
In practice this is only unmodded and offline.
Online adds a chunk of time to loading (and can be quite variable due to however konamis servers are).
IHs initial startup by itself pushes it past that.
So it just meant looking at a black screen, even though the logos aren't tied to loading timing it at least gives some sense of progress.

I played around with a replacement in light of the fact that it wasn't holding things up (in r90), where it would throw up random emblems, but ran in to issues with stability for some users (so I removed that in r115).
I was also bit wary that the nvidia logo was called by the exe rather than the scripts, in light of the above.

However your method pretty clear, and uses the new-game execution path which doesn't do logos in init (so it's a vanilla path rather than the kludge I had back when). So I'll add the option.
Cheers.

On my own end:

As some may have noticed, I've had more luck creating new fpkds, previously I had issues as the engine seems to require a specific file type order in fpkds or it will crash. fpks don't seem to have the same issue as far as I can tell.

This was acerbated by makebite just building fpks in whatever order it came across the files at.

My current workaround is to repack the fpkds after makebite with the gztool xml giving a specific file type order.
snakebite keeps the file order of packs it gets in the .mgsv

Since most of my packs I created by stripping down an existing pack, I've been using the type order from that is holding up.

I've thrown together a script to pull together the orders from the fpkd.xmls
https://mega.nz/#!jd1B3RCA!wh-pTVPBAT-y ... EdNM1eYzVU
And this is the nearest I can figure from that:
--tex reconstructed file type order from analysing output of this process.
--all positions reletively solid except for "bnd", analysis from my ExtensionOrder.lua puts it somewhere between veh and tgt.
--have put it between des and tgt in line with init.lua RegisterPackageExtensionInfo call
-- RegisterPackageExtensionInfo call seems to mostly match my derived order in reverse - however clo doesnt fit the order and lng isn't in its table.
local fpkdExtensionsOrder={ "fox2", "evf", "parts", "vfxlb", "vfx", "vfxlf", "veh", "frld", "des", "bnd", "tgt", "phsd", "ph", "sim", "clo", "fsd", "sdf", "lua", "lng" }
Should probably patch makebite at some point.

Apart from that, I'm hitting some overall gameobject limit, enough instances and the game gets crashy at different points (sometimes rolling into an area with the specific gameobjects, sometimes on checkpoint reload, sometimes on map exit).
This means I've had to rein back on my total count of helis,skulls,walkers,soldiers (since I can't run-time change the count and cant guarantee users don't have all enabled at once).
That's the issue we're hitting with IH and your animals mod morbidslinky, I haven't got round to atualy testing your mod with IH or even checked out what you have for your counts though sorry.

There's various references to memory allocation amounts both in the startup luas and in some fox2s, but my initial (brief) tinkering hasn't had any noticeable effect.

Posted: **Sat Nov 12, 2016 10:07 pm**

TruckerHatRyan wrote: Don't suppose anyone has found something that would explain how the game decides which models don't show up in first person when using the int scope?

Almost seems like it would be something contained within the .fmdl files that the game looks for, or maybe it could be related to .fv2 files somehow? Of course can use the arm slotting tricks to get around it but being able to prevent any model from showing up in first person would open up a lot of more possibilities with model swapping.
unknown123 wrote:You mean parts of player's model? I think it is hardcoded and depends on camera.
Yep that is what I was referring to, how on models that were not intended to be playable you will see them in first person when using the int scope if you make them the main .fmdl of an fpk.
.

I was of the same impression as you TruckerHatRyan - while the game clearly triggers it on camera, it seems like it's the equivalent of the fv2 hide model functionality, and the fmdl submodel hide functionality (as shown by the svs body models).
Otherwise if it was a hardcoded reference you'd think the head would still hide when the body model is the only thing that's been swapped?

Posted: **Sun Nov 13, 2016 12:09 am**

TruckerHatRyan wrote: Don't suppose anyone has found something that would explain how the game decides which models don't show up in first person when using the int scope?

Almost seems like it would be something contained within the .fmdl files that the game looks for, or maybe it could be related to .fv2 files somehow? Of course can use the arm slotting tricks to get around it but being able to prevent any model from showing up in first person would open up a lot of more possibilities with model swapping.

Tex wrote:I was of the same impression as you TruckerHatRyan - while the game clearly triggers it on camera, it seems like it's the equivalent of the fv2 hide model functionality, and the fmdl submodel hide functionality (as shown by the svs body models).
Otherwise if it was a hardcoded reference you'd think the head would still hide when the body model is the only thing that's been swapped?

Before YouRePrettyGood disappeared, he mentioned to me in a PM that he might've found an offset in the .fmdl files that determines whether a mesh disappears or not when entering first-person/the Int-Scope. Unfortunately, he didn't mention what the offset may have been and he hasn't posted since. Seems like it might be determined whether a mesh should disappear or not based on both the offset and camera distance.

On an unrelated note, I've made some some more progress on the .mtar format(s). I've separated the two different formats with different names. I call them Mtar Type 1 and Mtar Type 2. Mtar Type 2 is the more confusing format and still needs more research.

Code: Select all

Mtar Type 1 Format (i.e. TppPlayer2Facial.mtar) Info:
Uses Little-Endian
0x0 - 0x3 (32): Signature? 9C A8 FF 0B = Gzs, 72 2B 01 0C = Tpp
0x4 - 0x7 (32): Number of files. Each line starting at 0x20 seems to indicate a file
0x8 - 0xB (32): Unknown. TppGzPlayerLayers and player2_resident share bytes 12 00 38 00. TppGzPlayerFacial (26 00 44 00) and TppPlayer2Facial (23 00 44 00) slightly differ.
0xC - 0xF (32): Unknown. TppGzPlayerLayers and player2_resident share bytes 00 00 00 00. TppGzPlayerFacial and TppPlayer2Facial share bytes 06 00 01 00.
0x10 - 0x1F: Padding.

File declaring/listing begins at 0x20. All files in the list follow this format:
0x20 - 0x27: Unknown. Best guess is this might be the hashed name of the file. I.e. 63 0C 20 DA FA AF 53 = 3affada200c63
0x28 - 0x2B (32): File location offset.
0x2C - 0x2F (32): File length.

Gani Type 1 File Format:
0x0 - 0x4 (40): Signature (D2 A2 FC 0B 20).
0x5 - 0x7 (24): Padding.
0x8 - 0xB (32): File length.
0xC - 0xF (32): Padding.


Mtar Type 2 Format (i.e. player2_resident.mtar):
0x0 - 0x3 (32): Signature? 72 2B 01 0C = Tpp
0x4 - 0x7 (32): Number of files. Each 2 lines starting at 0x20 seems to indicate a file
0x8 - 0xB (32): Unknown. TppGzPlayerLayers and player2_resident share bytes 12 00 38 00. TppGzPlayerFacial (26 00 44 00) and TppPlayer2Facial (23 00 44 00) slightly differ.
0xC - 0xF (32): Unknown. TppGzPlayerLayers and player2_resident share bytes 00 00 00 00. TppGzPlayerFacial and TppPlayer2Facial share bytes 06 00 01 00.
0x10 - 0x13: Unknown.
0x14 - 0x17 (32): Offset for some kind of signature (EF AA BD 4F).
0x18 - 0x1F (80): Padding.

EF AA BD 4F seems to be a file header of some sort. On the line that contains this header, 0x8 - 0xB declares file length. Another file header appears after this file. The
following header can either be 4B 53 E4 91 or 84 77 9A 3B. More possibilities for this header may exist.

File declaring/listing begins at 0x20. All files in the list follow this format:
0x20 - 0x27: Unknown. Best guess is this might be the hashed name of the file. I.e. 63 0C 20 DA FA AF 53 = 3affada200c63
0x28 - 0x2B (32): File location offset.
0x2C - 0x2F (32): File length. A repeated pattern in this area seems to mean multiply by 0x10 to get the length. It also indicates that the file is extended(?).
0x30 - 0x31 (16): If the previous file was extended, there will be a value here. Multiply the value by 0x10 to get the extension length. The extension immediately follows the end of the extended file.
0x32 - 0x37 (48): Padding.
0x38 - 0x3B (32): Unknown:
0x3C - 0x3F (32): Padding.

I've attached a sample of what I believe are various files I've manually extracted from Mtar Type 2s. If anyone could take a look at them in a hex editor and see if they can notice anything related to the .fmdl format such as bones names, it would be a big help. Thanks!

Posted: **Sun Nov 13, 2016 12:33 am**

Tex wrote:I was of the same impression as you TruckerHatRyan - while the game clearly triggers it on camera, it seems like it's the equivalent of the fv2 hide model functionality, and the fmdl submodel hide functionality (as shown by the svs body models).
Otherwise if it was a hardcoded reference you'd think the head would still hide when the body model is the only thing that's been swapped?

BobDoleOwndU wrote:Before YouRePrettyGood disappeared, he mentioned to me in a PM that he might've found an offset in the .fmdl files that determines whether a mesh disappears or not when entering first-person/the Int-Scope. Unfortunately, he didn't mention what the offset may have been and he hasn't posted since. Seems like it might be determined whether a mesh should disappear or not based on both the offset and camera distance.

Thanks guys. I have sent YouRePrettyGood a PM just in case he returns. Not getting my hopes up, but with some luck maybe we can get this figured out.

Posted: **Sun Nov 13, 2016 4:09 am**

BobDoleOwndU wrote:On an unrelated note, I've made some some more progress on the .mtar format(s). I've separated the two different formats with different names. I call them Mtar Type 1 and Mtar Type 2. Mtar Type 2 is the more confusing format and still needs more research.
Code: Select all
*details*
I've attached a sample of what I believe are various files I've manually extracted from Mtar Type 2s. If anyone could take a look at them in a hex editor and see if they can notice anything related to the .fmdl format such as bones names, it would be a big help. Thanks!

In case it can be any help, here are some MGS4 MTAR + MTSQ sample files:
https://yadi.sk/d/iFCQjSzmyVVvv

XeNTaX

Metal Gear Solid 5 Ground Zeroes/Phantom Pain g0s archive

Re: Metal Gear Solid 5 Ground Zeroes/Phantom Pain g0s archiv

Re: Metal Gear Solid 5 Ground Zeroes/Phantom Pain g0s archiv

Re: Metal Gear Solid 5 Ground Zeroes/Phantom Pain g0s archiv

Re: Metal Gear Solid 5 Ground Zeroes/Phantom Pain g0s archiv

Re: Metal Gear Solid 5 Ground Zeroes/Phantom Pain g0s archiv

Re: Metal Gear Solid 5 Ground Zeroes/Phantom Pain g0s archiv

Re: Metal Gear Solid 5 Ground Zeroes/Phantom Pain g0s archiv

Re: Metal Gear Solid 5 Ground Zeroes/Phantom Pain g0s archiv

Re: Metal Gear Solid 5 Ground Zeroes/Phantom Pain g0s archiv

Re: Metal Gear Solid 5 Ground Zeroes/Phantom Pain g0s archiv

Re: Metal Gear Solid 5 Ground Zeroes/Phantom Pain g0s archiv

Re: Metal Gear Solid 5 Ground Zeroes/Phantom Pain g0s archiv

Re: Metal Gear Solid 5 Ground Zeroes/Phantom Pain g0s archiv

Re: Metal Gear Solid 5 Ground Zeroes/Phantom Pain g0s archiv

Re: Metal Gear Solid 5 Ground Zeroes/Phantom Pain g0s archiv