The current study reports on the fabrication of composite scaffolds based on polycaprolactone (PCL) and cerium (Ce)-containing powders, followed by their characterization from compositional, structural, morphological, optical and biological points of view. First, CeO2, Ce-doped calcium phosphates and Ce-substituted bioglass were synthesized by wet-chemistry methods (precipitation/coprecipitation and sol-gel) and subsequently loaded on PCL fibres processed by electrospinning. The powders were proven to be nanometric or micrometric, while the investigation of their phase composition showed that Ce was present as a dopant within the crystal lattice of the obtained calcium phosphates or as crystalline domains inside the glassy matrix. The best bioactivity was attained in the case of Ce-containing bioglass, while the most pronounced antibacterial effect was visible for Ce-doped calcium phosphates calcined at a lower temperature. The scaffolds were composed of either dimensionally homogeneous fibres or mixtures of fibres with a wide size distribution and beads of different shapes. In most cases, the increase in polymer concentration in the precursor solution ensured the achievement of more ordered fibre mats. The immersion in SBF for 28 days triggered an incipient degradation of PCL, evidenced mostly through cracks and gaps. In terms of biological properties, the composite scaffolds displayed a very good biocompatibility when tested with human osteoblast cells, with a superior response for the samples consisting of the polymer and Ce-doped calcium phosphates.