Inverted bulk heterojunction solar cells were fabricated using poly(3-hexylthiophene) (P3HT) blended with two different fullerene derivatives namely phenyl-C61-butyric acid methyl ester (PC60BM) and indene-C60 bis-adduct (IC60BA). The effects of annealing temperatures on the morphology, optical and structural properties were studied and correlated to differences in photovoltaic device performance. It was observed that annealing temperature significantly improved the performance of P3HT:IC60BA solar cells while P3HT:PC60BM cells showed relatively less improvement. The performance improvement is attributed to the extent of fullerene mixing with polymer domains. Energy filtered transmission electron microscopy (EFTEM) and x-ray diffraction (XRD) results showed that ICBA mixes with disordered P3HT much more readily than PC60BM which leads to lower short circuit current density and fill factor for P3HT:IC60BA cells annealed below 120 °C. Annealing above 120 °C improves the crystallinity of P3HT in case of P3HT:IC60BA whereas in P3HT:PC60BM films, annealing above 80 °C leads to negligible change in crystallinity. Crystallization of P3HT also leads to higher domain purity as seen EFTEM. Further it is seen that cells processed with additive nitrobenzene (NB) showed enhanced short circuit current density and power conversion efficiency regardless of the fullerene derivative used. Addition of NB led to nanoscale phase separation between purer polymer and fullerene domains. Kelvin probe force microscopy (KPFM) images showed that enhanced domain purity in additive casted films led to a sharper interface between polymer and fullerene. Enhanced domain purity and interfacial sharpness led to lower bimolecular recombination and higher mobility and charge carrier lifetime in NB modified devices.