For a series of six diketopyrrolopyrrole (DPP) based conjugated polymers we establish a direct correlation between their external quantum efficiencies (EQE) in organic solar cells and the fibrillar microstructure in the blend. The polymers consist of electron-deficient DPP units, carrying long branched 2'-decyltetradecyl (DT) side chains for solubility, that alternate along the main chain with electron-rich aromatic segments comprising benzene, thiophene, or fused aromatic rings. The high molecular weight DT-DPP polymers were incorporated in bulk heterojunction solar cells with [6,6]-phenyl-C71-butyric acid methyl ester ([70]PCBM) as acceptor. The morphology of the DT-DPP:[70]PCBM blends is characterized by a semi-crystalline fibrillar microstructure with fibril widths between 4.5 and 30 nm as evidenced from transmission electron microscopy. A clear correlation is found between the widths of the fibrils and the EQE for photon to electron conversion. The highest EQEs (60%) and power conversion efficiencies (7.1%) are obtained for polymers with fibril widths less than 12 nm. For blends with fibrils wider than 12 nm, the EQE is low because exciton diffusion becomes limiting for charge generation. Interestingly, the correlation found here matches with previous data on related DPP-based polymers. This suggests that for this class of materials the relation between fiber width and EQE is universal. The fiber width is largely correlated with the solubility of the polymers, with less soluble DPP-polymers giving narrower fibrils.