XeNTaX Forum Index
Forum MultiEx Commander Tools Tools Home
It is currently Tue May 30, 2017 2:05 pm

All times are UTC + 1 hour


Forum rules


Please click here to view the forum rules



Post new topic Reply to topic  [ 46 posts ]  Go to page 1, 2, 3, 4  Next
Author Message
 Post subject: Mafia 3 - .SDS
PostPosted: Thu Oct 06, 2016 8:21 pm 
Offline
advanced

Joined: Sat Oct 04, 2014 5:22 am
Posts: 47
Has thanked: 0 time
Have thanks: 2 times

seems to be using the same filetype.

You can make the ads go away by registering



Top
 Profile  
 
 Post subject: Re: Mafia 3 - .SDS
PostPosted: Fri Oct 07, 2016 2:24 am 
Offline
n00b

Joined: Mon Jan 08, 2007 12:47 am
Posts: 10
Has thanked: 0 time
Have thanks: 0 time
Sample file
http://www24.zippyshare.com/v/NpZjwaKD/file.html

SDS extractors for Mafia 2 don't work. File contains ZLIB, but most probably is just slightly different from older format.


Top
 Profile  
 
 Post subject: Re: Mafia 3 - .SDS
PostPosted: Fri Oct 07, 2016 8:17 am 
Offline
ultra-n00b

Joined: Thu Dec 03, 2015 6:20 pm
Posts: 9
Has thanked: 3 times
Have thanks: 0 time
Any news on this? We're thinking to translate this game to our language but we couldn't start to the project because lack of tools :)


Top
 Profile  
 
 Post subject: Re: Mafia 3 - .SDS
PostPosted: Fri Oct 07, 2016 7:55 pm 
Offline
ultra-n00b

Joined: Fri Jun 15, 2012 3:05 pm
Posts: 1
Has thanked: 0 time
Have thanks: 0 time
At a quick glance, they look extremely similar, it's version 13 (Mafia 2) vs 14 (Mafia 3). I don't have much/any knowledge of Mafia 2 modding or its files, but if you compare tables.sds of both games, the header seems to be of the same length and structure for the most part.

Shouldn't be too hard. With one of the old Mafia 2 SDS tools I got a hash error when hex editing the version of an .SDS, but I have no idea how flexible that tool was written.

Would take a shot at it myself, but don't think I have the time.


Top
 Profile  
 
 Post subject: Re: Mafia 3 - .SDS
PostPosted: Sat Oct 08, 2016 2:10 pm 
Offline
advanced
User avatar

Joined: Fri Aug 22, 2014 6:11 pm
Posts: 66
Location: Earth
Has thanked: 1 time
Have thanks: 0 time
UP :!:


Top
 Profile  
 
 Post subject: Re: Mafia 3 - .SDS
PostPosted: Sun Oct 09, 2016 12:01 am 
Offline
advanced
User avatar

Joined: Thu Mar 08, 2012 6:17 pm
Posts: 45
Location: California, USA
Has thanked: 6 times
Have thanks: 12 times
Posting my research here in case anyone can put it to good use. I'll probably make my own tools eventually, but I don't have an ETA on that :P

Thanks to aluigi for his Mafia .SDS (PS3) script, it helped point me in the right direction.

NOTE: I'm probably wrong on a lot of this, so please feel free to make corrections. I'm most likely off on the chunkCount being used to determine how many SDSChunkData's there are.

Code:
struct SDSFile {
    int magic; // 0x534453 ('SDS\0')
   
    int version; // 0x14 (v20 / v1.4?)
   
    enum SDSType : int {
        PC = 0x4350,
       
        /* These are ASSUMED, I only have the PC version */
        PS4 = 0x345350,
        XB1 = 0x314258,
    } type;
   
    int unk_0C; // 0x5DE53FDE
   
    int dataInfoOffset;
    int dataOffset;
   
    int unk_18; // always zero?
    int unk_1C; // offset / size of something?
    int unk_20; // always zero?
    int unk_24; // offset / size of something?
    int unk_28; // always zero?
    int unk_2C; // 1
    int unk_30; // always zero?
    int unk_34; // always zero?
    int unk_38; // always zero?
    int unk_3C; // always zero?
   
    int chunkCount; // how many data chunks? (educated guess)
    int reserved; // checksum, hash, or key of some sort?
   
    /* @ dataInfoOffset */
    struct SDSDataInfo {
        int count; // how many data types there are
       
        struct SDSDataType {
            int typeIndex; // local index to reference this type
           
            int strLen;
            char typeName[strLen]; // no null-terminator
           
            int reserved; // possibly a guard? (always zero?)
        } dataTypes[count];
   
    } dataInfo;
   
    /* @ dataOffset */
    struct SDSData {
        int magic; // 0x6C7A4555 ('UEzl')
       
        int unk_04; // 0x10000? (never seems to change)
        char unk_05; // 4?
       
        struct SDSChunkData {
            int dataSize; // size of SDSChunkData
            char dataType; // reference to SDSDataInfo.dataType
           
            struct SDSChunkData {
                int memorySize; // size in memory? (e.g. uncompressed size)
               
                int bufferOffset; // offset to start of data
               
                int memorySize_2; // SDSData.unk_04 again?
               
                short unk_0A; // number of something? (this format is confusing)
                short unk_0C; // 0x80F?
               
                short bufferSize; // this might be an int, but unlikely
                /*
                    <<< Data padding >>>
                */
                /* @ bufferOffset */
                char buffer[bufferSize];
            } data;
        } chunks[chunkCount];
    };
};


Top
 Profile  
 
 Post subject: Re: Mafia 3 - .SDS
PostPosted: Sun Oct 09, 2016 12:59 am 
Offline
advanced
User avatar

Joined: Fri Aug 22, 2014 6:11 pm
Posts: 66
Location: Earth
Has thanked: 1 time
Have thanks: 0 time
CarLuver69 wrote:
Posting my research here in case anyone can put it to good use. I'll probably make my own tools eventually, but I don't have an ETA on that :P

Thanks to aluigi for his Mafia .SDS (PS3) script, it helped point me in the right direction.

NOTE: I'm probably wrong on a lot of this, so please feel free to make corrections. I'm most likely off on the chunkCount being used to determine how many SDSChunkData's there are.


QuickBMS have letter limit sir, is there any solution except BMS script?


Top
 Profile  
 
 Post subject: Re: Mafia 3 - .SDS
PostPosted: Sun Oct 09, 2016 3:21 am 
Offline
advanced
User avatar

Joined: Thu Mar 08, 2012 6:17 pm
Posts: 45
Location: California, USA
Has thanked: 6 times
Have thanks: 12 times
Skrillex wrote:
QuickBMS have letter limit sir, is there any solution except BMS script?


Not at the moment sir, someone will need to make an SDS extractor. There are tools for Mafia II but they would need to be updated.

If I ever make my own tools, I'll post a link in this thread. But don't hold your breath for anything soon. :P


Top
 Profile  
 
 Post subject: Re: Mafia 3 - .SDS
PostPosted: Sun Oct 09, 2016 8:47 am 
Offline
ultra-n00b

Joined: Sat Oct 08, 2016 2:03 pm
Posts: 1
Has thanked: 0 time
Have thanks: 0 time
Anyone knows in game texts or subtitles are encrypted or plain?


Top
 Profile  
 
 Post subject: Re: Mafia 3 - .SDS
PostPosted: Sun Oct 09, 2016 10:57 am 
Offline
advanced

Joined: Thu Jul 12, 2012 12:24 pm
Posts: 61
Has thanked: 6 times
Have thanks: 15 times
Not encrypted. They are in readable form but with Binary tables. Fonts are GFX scaleform. Other words SWF.
EDiT: Correction, subtitles are encrypted, but stringtable doesn't.

Sounds can be extracted with Ravioli. It's a pck file with Wwise format.


Last edited by GRiNDERKILLER on Sun Oct 09, 2016 3:31 pm, edited 1 time in total.

Top
 Profile  
 
 Post subject: Re: Mafia 3 - .SDS
PostPosted: Sun Oct 09, 2016 2:56 pm 
Offline
veteran

Joined: Sat Dec 08, 2012 11:14 am
Posts: 149
Has thanked: 13 times
Have thanks: 2 times
somethink's about audio


Top
 Profile  
 
 Post subject: Re: Mafia 3 - .SDS
PostPosted: Tue Oct 11, 2016 8:38 pm 
Offline
ultra-n00b

Joined: Mon Jun 23, 2014 10:06 am
Posts: 3
Has thanked: 2 times
Have thanks: 3 times
Hey guys,

I've started to try porting Ricks Mafia 2 SDS Extractor tool to Mafia 3 (based on CarLuver69s source)
Its open source and can be found here: https://github.com/MartinJK/Mafia3SDSExplorer



When trying out a testfile, this was my result (It shows not available because I'm not able to extract the names of the resources at the moment)
Image

Looking deeper into it, I've tried to find the TEA stuff of Mafia 2 in Mafia 3, it seems that there's no more TEA encryption at all, also XML content in SSD files is gone (was plaintext in mafia 2's ssds)
Image

At the moment I'm dealing with an issue that the extractor crashes because of invalid memory stream / reading position.
Also the content seems to be encrypted, I've tried to open an encrypted DDS file -> exception.

Hope it helps someone here :)


Top
 Profile  
 
 Post subject: Re: Mafia 3 - .SDS
PostPosted: Tue Oct 11, 2016 8:44 pm 
Offline
advanced

Joined: Sun Aug 28, 2011 4:47 pm
Posts: 47
Has thanked: 22 times
Have thanks: 3 times
Aluigi seems to have made some tweaks to his quickbms script for PC version, not sure if it helps in any way: http://aluigi.altervista.org/bms/mafia_sds_ps3.bms

Your project looks awesome though, let's hope you make progress soon :)


Top
 Profile  
 
 Post subject: Re: Mafia 3 - .SDS
PostPosted: Tue Oct 11, 2016 10:27 pm 
Offline
advanced
User avatar

Joined: Fri Aug 22, 2014 6:11 pm
Posts: 66
Location: Earth
Has thanked: 1 time
Have thanks: 0 time
BitEmE wrote:
Hey guys,

I've started to try porting Ricks Mafia 2 SDS Extractor tool to Mafia 3 (based on CarLuver69s source)
Its open source and can be found here: https://github.com/MartinJK/Mafia3SDSExplorer

When trying out a testfile, this was my result (It shows not available because I'm not able to extract the names of the resources at the moment)


Looking deeper into it, I've tried to find the TEA stuff of Mafia 2 in Mafia 3, it seems that there's no more TEA encryption at all, also XML content in SSD files is gone (was plaintext in mafia 2's ssds)

At the moment I'm dealing with an issue that the extractor crashes because of invalid memory stream / reading position.
Also the content seems to be encrypted, I've tried to open an encrypted DDS file -> exception.

Hope it helps someone here :)


I hope you make it my friend. I'm counting on you.


Top
 Profile  
 
 Post subject: Re: Mafia 3 - .SDS
PostPosted: Wed Oct 12, 2016 1:07 am 
Offline
M-M-M-Monster veteran
M-M-M-Monster veteran

Joined: Wed Mar 31, 2010 6:54 am
Posts: 1592
Has thanked: 59 times
Have thanks: 741 times
BitEmE wrote:
Also the content seems to be encrypted, I've tried to open an encrypted DDS file -> exception.

They not encrypted, chunks are compressed with Zlib. About XML toc > not exist anymore. Content filenames hashed (fnv64a).

Code:
uint64 m3_gethash(unsigned char *m_String, int dwLength)
{
   uint64 dwHash = 0xCBF29CE484222325;
   for(int i = 0; i < dwLength; i++)
   {
      dwHash *= 0x00000100000001B3;
      dwHash ^= m_String[i];
   }
  return dwHash;
}


Some names

Code:
/gui/pop/garage.dds
/maps/engine/debug_font.dds
/maps/engine/env_brdf_lut_blinn.dds
/maps/engine/normalfix.dds
/maps/engine/pseudoshadow.dds
/maps/engine/sincos.dds
/maps/engine/smaa/area_tex_dx10.dds
/maps/engine/smaa/search_tex.dds
/maps/engine/water_normal_variance.dds
/scripts/common/base/butil.lua
/scripts/common/base/code_extensions.lua
/scripts/common/base/debug_utils.lua
/scripts/common/base/game_structure_console.lua
/scripts/common/base/gutil.lua
/scripts/common/base/script.lua
/scripts/common/base/script_classes.lua
/scripts/common/base/threads.lua
/scripts/common/base/various_utils.lua
/scripts/common/cityapscripts/animationlists.lua
/scripts/common/commands_library/common/common_init.lua
/scripts/common/commands_library/common/human/common_human.lua
/scripts/common/commands_library/common/human/common_human_events.lua
/scripts/common/commands_library/common/human/common_human_obsolete.lua
/scripts/common/commands_library/common/props.lua
/scripts/common/commands_library/common/vehicle/common_vehicle.lua
/scripts/common/commands_library/common/vehicle/common_vehicle_events.lua
/scripts/common/commands_library/common/vehicle/common_vehicle_obsolete.lua
/scripts/common/commands_library/entity/ab_detector.lua
/scripts/common/commands_library/entity/actor_detector.lua
/scripts/common/commands_library/entity/boat.lua
/scripts/common/commands_library/entity/camera.lua
/scripts/common/commands_library/entity/car/car.lua
/scripts/common/commands_library/entity/car/car_obsolete.lua
/scripts/common/commands_library/entity/crash_object.lua
/scripts/common/commands_library/entity/cutscene.lua
/scripts/common/commands_library/entity/distance_detector.lua
/scripts/common/commands_library/entity/door.lua
/scripts/common/commands_library/entity/fire_target.lua
/scripts/common/commands_library/entity/human/human.lua
/scripts/common/commands_library/entity/human/human_obsolete.lua
/scripts/common/commands_library/entity/interact_detector.lua
/scripts/common/commands_library/entity/light.lua
/scripts/common/commands_library/entity/particle.lua
/scripts/common/commands_library/entity/player.lua
/scripts/common/commands_library/entity/sensor.lua
/scripts/common/commands_library/entity/trigger_volume.lua
/scripts/common/commands_library/nodes_moved_to_code.lua
/scripts/common/commands_library/system/audio/game_audio.lua
/scripts/common/commands_library/system/audio/game_audio_deprecated.lua
/scripts/common/commands_library/system/audio/viscript_wwise_events.lua
/scripts/common/commands_library/system/audio/viscript_wwise_states.lua
/scripts/common/commands_library/system/game/capture.lua
/scripts/common/commands_library/system/game/collectibles.lua
/scripts/common/commands_library/system/game/coverage.lua
/scripts/common/commands_library/system/game/datastore.lua
/scripts/common/commands_library/system/game/delivery.lua
/scripts/common/commands_library/system/game/fleet.lua
/scripts/common/commands_library/system/game/fsm.lua
/scripts/common/commands_library/system/game/game.lua
/scripts/common/commands_library/system/game/garage.lua
/scripts/common/commands_library/system/game/hideout.lua
/scripts/common/commands_library/system/game/hud.lua
/scripts/common/commands_library/system/game/hud/hud_game_structure.lua
/scripts/common/commands_library/system/game/hud/objective_missions.lua
/scripts/common/commands_library/system/game/hud/objective_rackets.lua
/scripts/common/commands_library/system/game/hud/objective_spokes.lua
/scripts/common/commands_library/system/game/interactions.lua
/scripts/common/commands_library/system/game/persistent_location.lua
/scripts/common/commands_library/system/game/pipcamera.lua
/scripts/common/commands_library/system/game/police.lua
/scripts/common/commands_library/system/game/savesystem.lua
/scripts/common/commands_library/system/game/services.lua
/scripts/common/commands_library/system/game/shop.lua
/scripts/common/commands_library/system/game/sound.lua
/scripts/common/commands_library/system/game/stat_tracker.lua
/scripts/common/commands_library/system/game/streammap.lua
/scripts/common/commands_library/system/game/tailing.lua
/scripts/common/commands_library/system/game/traffic.lua
/scripts/common/commands_library/system/game/weather.lua
/scripts/common/commands_library/system/game/wiretapping.lua
/scripts/common/commands_library/system/game_director/activity.lua
/scripts/common/commands_library/system/game_director/business.lua
/scripts/common/commands_library/system/game_director/crime_family.lua
/scripts/common/commands_library/system/game_director/custom_state.lua
/scripts/common/commands_library/system/game_director/dialog.lua
/scripts/common/commands_library/system/game_director/economy.lua
/scripts/common/commands_library/system/game_director/kickback.lua
/scripts/common/commands_library/system/game_director/objective.lua
/scripts/common/commands_library/system/game_director/old.lua
/scripts/common/commands_library/system/game_director/other.lua
/scripts/common/commands_library/system/game_director/persistent_characters.lua
/scripts/common/commands_library/system/game_director/quest.lua
/scripts/common/commands_library/system/system.lua
/scripts/common/commands_library/system/test_nodes/bodyshop.lua
/scripts/common/commands_library/system/test_nodes/game_structure.lua
/scripts/common/commands_library/system/test_nodes/patrols_and_stalking.lua
/scripts/common/commands_library/system/test_nodes/testscenes.lua
/scripts/common/commands_library/system/test_nodes/user_bar.lua
/scripts/common/helpers/camera_helper.lua
/scripts/common/helpers/class.lua
/scripts/common/helpers/data/data_input_presets.lua
/scripts/common/helpers/data/data_journal.lua
/scripts/common/helpers/data/data_objectives.lua
/scripts/common/helpers/data/data_owas.lua
/scripts/common/helpers/data/data_traffic.lua
/scripts/common/helpers/data/data_tutorials.lua
/scripts/common/helpers/data/data_vo_selection.lua
/scripts/common/helpers/data/data_wiretapping.lua
/scripts/common/helpers/delivery_mission_defaults.lua
/scripts/common/helpers/delivery_objective_tables.lua
/scripts/common/helpers/difficulty_helper.lua
/scripts/common/helpers/dlc-services.lua
/scripts/common/helpers/entity_helper.lua
/scripts/common/helpers/event_helper.lua
/scripts/common/helpers/hud/skill_challenge_settings.lua
/scripts/common/helpers/hud/spoke_owa_icon_settings.lua
/scripts/common/helpers/mobile_store_inventory.lua
/scripts/common/helpers/persistent_locations/persistent_location_districts.lua
/scripts/common/helpers/persistent_locations/persistent_location_list.lua
/scripts/common/helpers/persistent_locations/persistent_location_teplates.lua
/scripts/common/helpers/services_definitions.lua
/scripts/common/helpers/tgrunwald/containers.lua
/scripts/common/helpers/tgrunwald/death_pool.lua
/scripts/common/helpers/tgrunwald/home_enter.lua
/scripts/common/helpers/tgrunwald/quest_helper.lua
/scripts/common/helpers/timer.lua
/scripts/common/helpers/viscript_helper.lua
/scripts/common/poi/poi.lua
/scripts/common/sandbox/gangsters_anger_manager.lua
/scripts/common/searchactionpoints/example.lua
/scripts/entities/interactions/interaction_placeholder_hold.vi
/scripts/entities/interactions/int_barrel_tossing_slave_hold.vi
/scripts/entities/interactions/int_construction_site_execution_boss_hold.vi
/scripts/entities/interactions/int_construction_site_execution_gunman_hold.vi
/scripts/entities/interactions/int_construction_site_execution_victim_hold.vi
/scripts/entities/interactions/int_father_james_serving_soup_civ_hold.vi
/scripts/entities/interactions/int_french_ward_brothel_fem_standing_rail_hold.vi
/scripts/entities/interactions/int_heist_giorgi_turn_in_shotgun_hold.vi
/scripts/entities/interactions/int_junky_hallucinogens_hold.vi
/scripts/entities/interactions/int_junky_kicked_hold.vi
/scripts/entities/interactions/int_junky_twitchy_hold.vi
/scripts/entities/interactions/int_kill_olivia_game_room_guard_l_hold.vi
/scripts/entities/interactions/int_kill_the_judge_dealer_1_hold.vi
/scripts/entities/interactions/int_kill_the_judge_dealer_2_hold.vi
/scripts/entities/interactions/int_kill_tommy_trainer_tending_hero_trainer_hold.vi
/scripts/entities/interactions/int_mugging_civilian_hold.vi
/scripts/entities/interactions/int_patrol_delay_eat_snack_hold.vi
/scripts/entities/interactions/int_plow_house_pull_guy_in_room_fem_hold.vi
/scripts/entities/interactions/int_plow_house_pull_guy_in_room_male_hold.vi
/scripts/entities/interactions/int_police_harass_hobo_hold.vi
/scripts/entities/interactions/int_police_obstruct_shop_hold.vi
/scripts/entities/interactions/int_prostitution_solicit_attempt_hold.vi
/scripts/entities/interactions/int_raking_working_hold.vi
/scripts/entities/interactions/int_standing_idle_hold.vi
/scripts/entities/interactions/int_standing_read_newspaper_hold.vi
/scripts/entities/interactions/int_voter_registration_police_disrupt_civ1_hold.vi
/scripts/entities/interactions/int_voter_registration_police_disrupt_civ2_hold.vi
/scripts/entities/interactions/int_west_marais_brothel_fem_standing_hold.vi
/scripts/entities/interactions/int_west_marais_brothel_pimp_standing_hold.vi
/scripts/entities/interactions/sc_protest_listener_hold.vi
/scripts/missions/bc_city/rare_cars.lua
/scripts/missions/statistic_tracking/statistic_tracking.lua


For DDS you need skip additional header with 0x2F bytes after decompress. You can do it here in TextureViewer:

Code:
      private void UpdatePreview()
      {
         MemoryStream memory = new MemoryStream();
         memory.Write(this.Resource.Data, 0, this.Resource.Data.Length);
         memory.Position = 0L;
         DDSFile expr_32 = new DDSFile();
         expr_32.Deserialize(memory);
         Image image = expr_32.Image(true, true, true, this.toggleAlphaButton.Checked);
         if (this.toolStripButton1.Checked)
         {
            this.previewPictureBox.Dock = DockStyle.Fill;
            this.previewPictureBox.Image = image;
            this.previewPictureBox.SizeMode = PictureBoxSizeMode.Zoom;
            return;
         }
         this.previewPictureBox.Dock = DockStyle.None;
         this.previewPictureBox.Image = image;
         this.previewPictureBox.Width = image.Width;
         this.previewPictureBox.Height = image.Height;
         this.previewPictureBox.SizeMode = PictureBoxSizeMode.Normal;
      }


Try set to 0x2F for code (memory.Position = 0L;) and this will be enought i guess :)



Last edited by Ekey on Wed Oct 12, 2016 11:15 am, edited 1 time in total.

Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 46 posts ]  Go to page 1, 2, 3, 4  Next

All times are UTC + 1 hour


Who is online

Users browsing this forum: No registered users and 6 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group