In this work, cyclic loading experiments were performed to investigate the influence of the elastic–plastic properties of the substrate material as well as of artificial brazing defects on the fatigue behavior of brazed steel joints. Brazing was performed with the steel X3CrNiMo 13-4 (AISI 316 C-NM) as a base material and with AuNi18 as a filler metal. After brazing, subsequent heat treatments were performed to influence the elastic–plastic properties of the substrate material. To investigate the influence of defects, defined defect geometries were introduced in the braze layer by electrical discharge machining (EDM). The results of the experiments show that the heat treatment has a significant influence on the fatigue lifetime. Furthermore it could be shown, that the fatigue behavior of brazed components is significantly influenced by the interaction of defects and elastic–plastic properties of the substrate material. To investigate the cyclic deformation, the materials strain response was investigated. Besides extensometer measurements, Digital Image Correlations (DIC) were used to measure the strain distribution during cyclic loading with a sufficient lateral resolution. Additional SEM investigations and FE-calculations were carried out to verify the experimental results and to investigate the complex mechanisms that can lead to fatigue failure of brazed components in detail. The results of this work underline the significant influence of the elastic–plastic properties of base material on the fatigue behavior and can be used for the appropriate design of brazed components.