The multi-scan resting state fMRI (rs-fMRI) dataset was recently released; thus the test-retest (TRT) reliability of rs-fMRI measures can be assessed. However, because this dataset was acquired only from a single group under a single condition, we cannot directly evaluate whether the rs-fMRI measures can generate reproducible between-condition or between-group results. Because the modulation of resting state activity has gained increasing attention, it is important to know whether one rs-fMRI metric can reliably detect the alteration of the resting activity. Here, we shared a public Eyes-Open (EO)/Eyes-Closed (EC) dataset for evaluating the split-half reproducibility of the rs-fMRI measures in detecting changes of the resting state activity between EO and EC. As examples, we assessed the split-half reproducibility of three widely applied rs-fMRI metrics: amplitude of low frequency fluctuation, regional homogeneity, and seed-based correlation analysis. Our results demonstrated that reproducible patterns of EO-EC differences can be detected by all three measures, suggesting the feasibility of the EO/EC dataset for performing reproducibility assessment for other rs-fMRI measures.