Mafia 3 - .SDS

[iambosh]

seems to be using the same filetype.

[Espio]

Sample file
http://www24.zippyshare.com/v/NpZjwaKD/file.html

SDS extractors for Mafia 2 don't work. File contains ZLIB, but most probably is just slightly different from older format.

[tice17]

Any news on this? We're thinking to translate this game to our language but we couldn't start to the project because lack of tools

[Timbab]

At a quick glance, they look extremely similar, it's version 13 (Mafia 2) vs 14 (Mafia 3). I don't have much/any knowledge of Mafia 2 modding or its files, but if you compare tables.sds of both games, the header seems to be of the same length and structure for the most part.

Shouldn't be too hard. With one of the old Mafia 2 SDS tools I got a hash error when hex editing the version of an .SDS, but I have no idea how flexible that tool was written.

Would take a shot at it myself, but don't think I have the time.

[Skrillex]

UP

[CarLuver69]

Posting my research here in case anyone can put it to good use. I'll probably make my own tools eventually, but I don't have an ETA on that

Thanks to aluigi for his Mafia .SDS (PS3) script, it helped point me in the right direction.

NOTE: I'm probably wrong on a lot of this, so please feel free to make corrections. I'm most likely off on the chunkCount being used to determine how many SDSChunkData's there are.

Code: Select all

struct SDSFile {
    int magic; // 0x534453 ('SDS\0')
    
    int version; // 0x14 (v20 / v1.4?)
    
    enum SDSType : int {
        PC = 0x4350,
        
        /* These are ASSUMED, I only have the PC version */
        PS4 = 0x345350,
        XB1 = 0x314258,
    } type;
    
    int unk_0C; // 0x5DE53FDE
    
    int dataInfoOffset;
    int dataOffset;
    
    int unk_18; // always zero?
    int unk_1C; // offset / size of something?
    int unk_20; // always zero?
    int unk_24; // offset / size of something?
    int unk_28; // always zero?
    int unk_2C; // 1
    int unk_30; // always zero?
    int unk_34; // always zero?
    int unk_38; // always zero?
    int unk_3C; // always zero?
    
    int chunkCount; // how many data chunks? (educated guess)
    int reserved; // checksum, hash, or key of some sort?
    
    /* @ dataInfoOffset */
    struct SDSDataInfo {
        int count; // how many data types there are
        
        struct SDSDataType {
            int typeIndex; // local index to reference this type
            
            int strLen;
            char typeName[strLen]; // no null-terminator
            
            int reserved; // possibly a guard? (always zero?)
        } dataTypes[count];
    
    } dataInfo;
    
    /* @ dataOffset */
    struct SDSData {
        int magic; // 0x6C7A4555 ('UEzl')
        
        int unk_04; // 0x10000? (never seems to change)
        char unk_05; // 4?
        
        struct SDSChunkData {
            int dataSize; // size of SDSChunkData
            char dataType; // reference to SDSDataInfo.dataType
            
            struct SDSChunkData {
                int memorySize; // size in memory? (e.g. uncompressed size)
                
                int bufferOffset; // offset to start of data
                
                int memorySize_2; // SDSData.unk_04 again?
                
                short unk_0A; // number of something? (this format is confusing)
                short unk_0C; // 0x80F?
                
                short bufferSize; // this might be an int, but unlikely
                /*
                    <<< Data padding >>>
                */
                /* @ bufferOffset */
                char buffer[bufferSize];
            } data;
        } chunks[chunkCount];
    };
};

[Skrillex]

Posting my research here in case anyone can put it to good use. I'll probably make my own tools eventually, but I don't have an ETA on that

Thanks to aluigi for his Mafia .SDS (PS3) script, it helped point me in the right direction.

NOTE: I'm probably wrong on a lot of this, so please feel free to make corrections. I'm most likely off on the chunkCount being used to determine how many SDSChunkData's there are.

QuickBMS have letter limit sir, is there any solution except BMS script?

[CarLuver69]

QuickBMS have letter limit sir, is there any solution except BMS script?

Not at the moment sir, someone will need to make an SDS extractor. There are tools for Mafia II but they would need to be updated.

If I ever make my own tools, I'll post a link in this thread. But don't hold your breath for anything soon.

[grimmp]

Anyone knows in game texts or subtitles are encrypted or plain?

[GRiNDERKILLER]

Not encrypted. They are in readable form but with Binary tables. Fonts are GFX scaleform. Other words SWF.
EDiT: Correction, subtitles are encrypted, but stringtable doesn't.

Sounds can be extracted with Ravioli. It's a pck file with Wwise format.

[kilik]

somethink's about audio

[BitEmE]

Hey guys,

I've started to try porting Ricks Mafia 2 SDS Extractor tool to Mafia 3 (based on CarLuver69s source)
Its open source and can be found here: https://github.com/MartinJK/Mafia3SDSExplorer



When trying out a testfile, this was my result (It shows not available because I'm not able to extract the names of the resources at the moment)


Looking deeper into it, I've tried to find the TEA stuff of Mafia 2 in Mafia 3, it seems that there's no more TEA encryption at all, also XML content in SSD files is gone (was plaintext in mafia 2's ssds)


At the moment I'm dealing with an issue that the extractor crashes because of invalid memory stream / reading position.
Also the content seems to be encrypted, I've tried to open an encrypted DDS file -> exception.

Hope it helps someone here

[ner0]

Aluigi seems to have made some tweaks to his quickbms script for PC version, not sure if it helps in any way: http://aluigi.altervista.org/bms/mafia_sds_ps3.bms

Your project looks awesome though, let's hope you make progress soon

[Skrillex]

Hey guys,

I've started to try porting Ricks Mafia 2 SDS Extractor tool to Mafia 3 (based on CarLuver69s source)
Its open source and can be found here: https://github.com/MartinJK/Mafia3SDSExplorer

When trying out a testfile, this was my result (It shows not available because I'm not able to extract the names of the resources at the moment)


Looking deeper into it, I've tried to find the TEA stuff of Mafia 2 in Mafia 3, it seems that there's no more TEA encryption at all, also XML content in SSD files is gone (was plaintext in mafia 2's ssds)

At the moment I'm dealing with an issue that the extractor crashes because of invalid memory stream / reading position.
Also the content seems to be encrypted, I've tried to open an encrypted DDS file -> exception.

Hope it helps someone here

I hope you make it my friend. I'm counting on you.

[Ekey]

Also the content seems to be encrypted, I've tried to open an encrypted DDS file -> exception.

They not encrypted, chunks are compressed with Zlib. About XML toc > not exist anymore. Content filenames hashed (fnv64a).

Code: Select all

uint64 m3_gethash(unsigned char *m_String, int dwLength)
{
   uint64 dwHash = 0xCBF29CE484222325;
   for(int i = 0; i < dwLength; i++)
   {
      dwHash *= 0x00000100000001B3;
      dwHash ^= m_String[i];
   }
  return dwHash;
}

Some names

Code: Select all

/gui/pop/garage.dds
/maps/engine/debug_font.dds
/maps/engine/env_brdf_lut_blinn.dds
/maps/engine/normalfix.dds
/maps/engine/pseudoshadow.dds
/maps/engine/sincos.dds
/maps/engine/smaa/area_tex_dx10.dds
/maps/engine/smaa/search_tex.dds
/maps/engine/water_normal_variance.dds
/scripts/common/base/butil.lua
/scripts/common/base/code_extensions.lua
/scripts/common/base/debug_utils.lua
/scripts/common/base/game_structure_console.lua
/scripts/common/base/gutil.lua
/scripts/common/base/script.lua
/scripts/common/base/script_classes.lua
/scripts/common/base/threads.lua
/scripts/common/base/various_utils.lua
/scripts/common/cityapscripts/animationlists.lua
/scripts/common/commands_library/common/common_init.lua
/scripts/common/commands_library/common/human/common_human.lua
/scripts/common/commands_library/common/human/common_human_events.lua
/scripts/common/commands_library/common/human/common_human_obsolete.lua
/scripts/common/commands_library/common/props.lua
/scripts/common/commands_library/common/vehicle/common_vehicle.lua
/scripts/common/commands_library/common/vehicle/common_vehicle_events.lua
/scripts/common/commands_library/common/vehicle/common_vehicle_obsolete.lua
/scripts/common/commands_library/entity/ab_detector.lua
/scripts/common/commands_library/entity/actor_detector.lua
/scripts/common/commands_library/entity/boat.lua
/scripts/common/commands_library/entity/camera.lua
/scripts/common/commands_library/entity/car/car.lua
/scripts/common/commands_library/entity/car/car_obsolete.lua
/scripts/common/commands_library/entity/crash_object.lua
/scripts/common/commands_library/entity/cutscene.lua
/scripts/common/commands_library/entity/distance_detector.lua
/scripts/common/commands_library/entity/door.lua
/scripts/common/commands_library/entity/fire_target.lua
/scripts/common/commands_library/entity/human/human.lua
/scripts/common/commands_library/entity/human/human_obsolete.lua
/scripts/common/commands_library/entity/interact_detector.lua
/scripts/common/commands_library/entity/light.lua
/scripts/common/commands_library/entity/particle.lua
/scripts/common/commands_library/entity/player.lua
/scripts/common/commands_library/entity/sensor.lua
/scripts/common/commands_library/entity/trigger_volume.lua
/scripts/common/commands_library/nodes_moved_to_code.lua
/scripts/common/commands_library/system/audio/game_audio.lua
/scripts/common/commands_library/system/audio/game_audio_deprecated.lua
/scripts/common/commands_library/system/audio/viscript_wwise_events.lua
/scripts/common/commands_library/system/audio/viscript_wwise_states.lua
/scripts/common/commands_library/system/game/capture.lua
/scripts/common/commands_library/system/game/collectibles.lua
/scripts/common/commands_library/system/game/coverage.lua
/scripts/common/commands_library/system/game/datastore.lua
/scripts/common/commands_library/system/game/delivery.lua
/scripts/common/commands_library/system/game/fleet.lua
/scripts/common/commands_library/system/game/fsm.lua
/scripts/common/commands_library/system/game/game.lua
/scripts/common/commands_library/system/game/garage.lua
/scripts/common/commands_library/system/game/hideout.lua
/scripts/common/commands_library/system/game/hud.lua
/scripts/common/commands_library/system/game/hud/hud_game_structure.lua
/scripts/common/commands_library/system/game/hud/objective_missions.lua
/scripts/common/commands_library/system/game/hud/objective_rackets.lua
/scripts/common/commands_library/system/game/hud/objective_spokes.lua
/scripts/common/commands_library/system/game/interactions.lua
/scripts/common/commands_library/system/game/persistent_location.lua
/scripts/common/commands_library/system/game/pipcamera.lua
/scripts/common/commands_library/system/game/police.lua
/scripts/common/commands_library/system/game/savesystem.lua
/scripts/common/commands_library/system/game/services.lua
/scripts/common/commands_library/system/game/shop.lua
/scripts/common/commands_library/system/game/sound.lua
/scripts/common/commands_library/system/game/stat_tracker.lua
/scripts/common/commands_library/system/game/streammap.lua
/scripts/common/commands_library/system/game/tailing.lua
/scripts/common/commands_library/system/game/traffic.lua
/scripts/common/commands_library/system/game/weather.lua
/scripts/common/commands_library/system/game/wiretapping.lua
/scripts/common/commands_library/system/game_director/activity.lua
/scripts/common/commands_library/system/game_director/business.lua
/scripts/common/commands_library/system/game_director/crime_family.lua
/scripts/common/commands_library/system/game_director/custom_state.lua
/scripts/common/commands_library/system/game_director/dialog.lua
/scripts/common/commands_library/system/game_director/economy.lua
/scripts/common/commands_library/system/game_director/kickback.lua
/scripts/common/commands_library/system/game_director/objective.lua
/scripts/common/commands_library/system/game_director/old.lua
/scripts/common/commands_library/system/game_director/other.lua
/scripts/common/commands_library/system/game_director/persistent_characters.lua
/scripts/common/commands_library/system/game_director/quest.lua
/scripts/common/commands_library/system/system.lua
/scripts/common/commands_library/system/test_nodes/bodyshop.lua
/scripts/common/commands_library/system/test_nodes/game_structure.lua
/scripts/common/commands_library/system/test_nodes/patrols_and_stalking.lua
/scripts/common/commands_library/system/test_nodes/testscenes.lua
/scripts/common/commands_library/system/test_nodes/user_bar.lua
/scripts/common/helpers/camera_helper.lua
/scripts/common/helpers/class.lua
/scripts/common/helpers/data/data_input_presets.lua
/scripts/common/helpers/data/data_journal.lua
/scripts/common/helpers/data/data_objectives.lua
/scripts/common/helpers/data/data_owas.lua
/scripts/common/helpers/data/data_traffic.lua
/scripts/common/helpers/data/data_tutorials.lua
/scripts/common/helpers/data/data_vo_selection.lua
/scripts/common/helpers/data/data_wiretapping.lua
/scripts/common/helpers/delivery_mission_defaults.lua
/scripts/common/helpers/delivery_objective_tables.lua
/scripts/common/helpers/difficulty_helper.lua
/scripts/common/helpers/dlc-services.lua
/scripts/common/helpers/entity_helper.lua
/scripts/common/helpers/event_helper.lua
/scripts/common/helpers/hud/skill_challenge_settings.lua
/scripts/common/helpers/hud/spoke_owa_icon_settings.lua
/scripts/common/helpers/mobile_store_inventory.lua
/scripts/common/helpers/persistent_locations/persistent_location_districts.lua
/scripts/common/helpers/persistent_locations/persistent_location_list.lua
/scripts/common/helpers/persistent_locations/persistent_location_teplates.lua
/scripts/common/helpers/services_definitions.lua
/scripts/common/helpers/tgrunwald/containers.lua
/scripts/common/helpers/tgrunwald/death_pool.lua
/scripts/common/helpers/tgrunwald/home_enter.lua
/scripts/common/helpers/tgrunwald/quest_helper.lua
/scripts/common/helpers/timer.lua
/scripts/common/helpers/viscript_helper.lua
/scripts/common/poi/poi.lua
/scripts/common/sandbox/gangsters_anger_manager.lua
/scripts/common/searchactionpoints/example.lua
/scripts/entities/interactions/interaction_placeholder_hold.vi
/scripts/entities/interactions/int_barrel_tossing_slave_hold.vi
/scripts/entities/interactions/int_construction_site_execution_boss_hold.vi
/scripts/entities/interactions/int_construction_site_execution_gunman_hold.vi
/scripts/entities/interactions/int_construction_site_execution_victim_hold.vi
/scripts/entities/interactions/int_father_james_serving_soup_civ_hold.vi
/scripts/entities/interactions/int_french_ward_brothel_fem_standing_rail_hold.vi
/scripts/entities/interactions/int_heist_giorgi_turn_in_shotgun_hold.vi
/scripts/entities/interactions/int_junky_hallucinogens_hold.vi
/scripts/entities/interactions/int_junky_kicked_hold.vi
/scripts/entities/interactions/int_junky_twitchy_hold.vi
/scripts/entities/interactions/int_kill_olivia_game_room_guard_l_hold.vi
/scripts/entities/interactions/int_kill_the_judge_dealer_1_hold.vi
/scripts/entities/interactions/int_kill_the_judge_dealer_2_hold.vi
/scripts/entities/interactions/int_kill_tommy_trainer_tending_hero_trainer_hold.vi
/scripts/entities/interactions/int_mugging_civilian_hold.vi
/scripts/entities/interactions/int_patrol_delay_eat_snack_hold.vi
/scripts/entities/interactions/int_plow_house_pull_guy_in_room_fem_hold.vi
/scripts/entities/interactions/int_plow_house_pull_guy_in_room_male_hold.vi
/scripts/entities/interactions/int_police_harass_hobo_hold.vi
/scripts/entities/interactions/int_police_obstruct_shop_hold.vi
/scripts/entities/interactions/int_prostitution_solicit_attempt_hold.vi
/scripts/entities/interactions/int_raking_working_hold.vi
/scripts/entities/interactions/int_standing_idle_hold.vi
/scripts/entities/interactions/int_standing_read_newspaper_hold.vi
/scripts/entities/interactions/int_voter_registration_police_disrupt_civ1_hold.vi
/scripts/entities/interactions/int_voter_registration_police_disrupt_civ2_hold.vi
/scripts/entities/interactions/int_west_marais_brothel_fem_standing_hold.vi
/scripts/entities/interactions/int_west_marais_brothel_pimp_standing_hold.vi
/scripts/entities/interactions/sc_protest_listener_hold.vi
/scripts/missions/bc_city/rare_cars.lua
/scripts/missions/statistic_tracking/statistic_tracking.lua

For DDS you need skip additional header with 0x2F bytes after decompress. You can do it here in TextureViewer:

Code: Select all

		private void UpdatePreview()
		{
			MemoryStream memory = new MemoryStream();
			memory.Write(this.Resource.Data, 0, this.Resource.Data.Length);
			memory.Position = 0L;
			DDSFile expr_32 = new DDSFile();
			expr_32.Deserialize(memory);
			Image image = expr_32.Image(true, true, true, this.toggleAlphaButton.Checked);
			if (this.toolStripButton1.Checked)
			{
				this.previewPictureBox.Dock = DockStyle.Fill;
				this.previewPictureBox.Image = image;
				this.previewPictureBox.SizeMode = PictureBoxSizeMode.Zoom;
				return;
			}
			this.previewPictureBox.Dock = DockStyle.None;
			this.previewPictureBox.Image = image;
			this.previewPictureBox.Width = image.Width;
			this.previewPictureBox.Height = image.Height;
			this.previewPictureBox.SizeMode = PictureBoxSizeMode.Normal;
		}

Try set to 0x2F for code (memory.Position = 0L;) and this will be enought i guess