Objective The objective of this study was to evaluate the potential impact of irreversible damage accrual in women with systemic lupus erythematosus (SLE) and adverse maternal and/or fetal/neonatal outcomes. Methods Retrospective cohort study with SLE pregnant patients was carried out from January 2011 to January 2020 at the Hospital University Pedro Ernesto (HUPE) of the State University of Rio de Janeiro, Brazil. Irreversible damage was defined according to SLICC/ACR damage index (SDI). The association of SDI on pregnancy outcomes was established by univariate and multivariate regression models and included demographic and clinical variables. Results This study included data from 260 patients in their first pregnancies after SLE diagnosis, with a quarter of them (67/260) scoring one or more points on SDI at the beginning of prenatal care. These patients presented more frequently adverse maternal events, namely, disease activity during pregnancy ( p = 0.004) and puerperium ( p = 0.001), active lupus nephritis ( p = 0.04), and hospitalizations ( p = 0.004), than those with no SDI score. Similarly, the risks of adverse fetal and neonatal outcomes were also higher among the patients with SDI ≥ 1 (59.7% vs 38.3% p = 0.001) even after controlling data for disease activity (SLEPDAI > 4). Patients with SDI ≥ 1 presented more frequently preterm deliveries (46.3% vs 31.6%; p = 0.01), small for gestational age infants (28.3% vs 18.1%; p = 0.04), and neonatal intensive care unit admission (26.9% vs 1.5%; p < 0.001). The multivariate analyses showed that SDI ≥ 1 is an independent risk factor for hospitalization due to obstetric complications ( p = 0.0008) and preterm delivery ( p = 0.009). Conclusion Pregnant SLE patients who present irreversible damage accrual may have higher risk of maternal and fetal adverse outcomes, independently of disease activity. These results should be validated in further prospective studies.