Detailed mineralogical and geochemical characteristics of typical surface sediments and hydrothermal deposits collected from the ultraslow-spreading Southwest Indian Ridge (SWIR) were studied by high-resolution XRD, SEM-EDS, XRF, and ICP-MS. The SWIR marine samples can be generally classified into two main categories: surface sediment (biogenic, volcanic) and hydrothermal-derived deposit; moreover, the surface sediment can be further classified into metalliferous and non-metalliferous based on the metalliferous sediment index (MSI). The chemical composition of biogenic sediment (mainly biogenic calcite) was characterized by elevated contents of Ca, Ba, Rb, Sr, Th, and light rare earth elements (LREE), while volcanic sediment (mainly volcanogenic debris) was relatively enriched in Mn, Mg, Al, Si, Ni, Cr, and high field strength elements (HFSEs). By contrast, the hydrothermal-derived deposit (mainly pyrite-marcasite, chalcopyrite-isocubanite, and low-temperature cherts) contained significantly higher contents of Fe, Cu, Zn, Pb, Mn, Co, Mo, Ag, and U. In addition, the metalliferous surface sediment contained a higher content of Cu, Mn, Fe, Co, Mo, Ba, and As. Compared with their different host (source) rock, the basalt-hosted marine sediments contained higher contents of Ti–Al–Zr–Sc–Hf and/or Mo–Ba–Ag; In contrast, the peridotite-hosted marine sediments were typically characterized by elevated concentrations of Mg–Cu–Ni–Cr and/or Co–Sn–Au. The differences in element enrichment and mineral composition between these sediment types were closely related to their sedimentary environments (e.g., near/far away from the vent sites) and inherited from their host (source) rock. Together with combinations of certain characteristic elements (such as Al–Fe–Mn and Si–Al–Mg), relict hydrothermal products, and diagnostic mineral tracers (e.g., nontronite, SiO2(bio), olivine, serpentine, talc, sepiolite, pyroxene, zeolite, etc.), it would be more effective to differentiate the host rock of deep-sea sediments and to detect a possible hydrothermal input.