Concerning the microstructure of cement based materials, the interfacial transition zone (ITZ) is regarded as a key feature for durability. In order to determine the influence of ITZ on the chloride diffusion, two series of experiments were carried out by varying both the volume fraction and the size of aggregate. The transport process of chloride is only diffusion in this work. Results show that the addition of aggregate tends to modify the pore structure of cement-based materials and the chloride diffusion process. When ITZ quantities increase with a decrease in grain size, or the aggregates are sufficiently closely-spaced that the adjacent ITZ overlap in space, the ITZ effect would dominate the effects of the blocking aggregate and the tortuosity of matrix paste, and as a result, fluid and ions transport will be significantly enhanced. Additionally, the binding capacity is higher when the ITZ content is high.