Mineral exploiting information is an important indicator to reflect regional mineral activities. Accurate extraction of this information is essential to mineral management and environmental protection. In recent years, there are an increasingly large number of pieces of research on land surface information classification by conducting multi-source remote sensing data. However, in order to achieve the best classification result, how to select the optimal feature combination is the key issue. This study creatively combines Out of Bag data with Recursive Feature Elimination (OOB RFE) to optimize the feature combination of the mineral exploiting information of non-metallic building materials in Fujian province, China. We acquired and integrated Ziyuan-1-02D (ZY-1-02D) hyperspectral imagery, landsat-8 multispectral imagery, and Sentinel-1 Synthetic Aperture Radar (SAR) imagery to gain spectrum, heat, polarization, and texture features; also, two machine learning methods were adopted to classify the mineral exploiting information in our study area. After assessment and comparison on accuracy, it proves that the classification generated from our new OOB RFE method, which combine with random forest (RF), can achieve the highest overall accuracy 93.64% (with a kappa coefficient of 0.926). Comparing with Recursive Feature Elimination (RFE) alone, OOB REF can precisely filter the feature combination and lead to optimal result. Under the same feature scheme, RF is effective on classifying the mineral exploiting information of the research field. The feature optimization method and optimal feature combination proposed in our study can provide technical support and theoretical reference for extraction and classification of mineral exploiting information applied in other regions.