Layered Bi-misfit cobaltates, such as Bi2Sr2Co2Oy, are the natural superlattice of an electrically insulating rocksalt (RS) type Bi2Sr2O4 layer and electrically conducting CoO2 layer, stacked along the crystallographic c-axis. RS and CoO2 layers are related through charge compensation reactions (or charge transfer). Therefore, thermoelectric transport properties are affected when doping or substitution is carried out in the RS layer. In this work, we have shown improved thermoelectric properties of spark plasma sintered Bi2Sr2-xCaxCo2Oy alloys (x = 0, 0.3 and 0.5). The substitution of Ca atoms affects the thermal properties by introducing point-defect phonon scattering, while the electronic conductivity and thermopower remain unaltered.