The relation between the microstructure and mechanical properties of the Fe–Nb dissimilar joint were investigated using nanoindentation. The weld metal consists mainly of Fe2Nb, α-Fe + Fe2Nb, Nb (s,s) and Fe7Nb6 phases. Radial cracks initiate from the corners of the impressions on the Fe2Nb phase (~20.5 GPa) when subjected to a peak load of 300 mN, whereas the fine lamellar structures (α-Fe + Fe2Nb) with an average hardness of 6.5 GPa are free from cracks. The calculated fracture toughness of the Fe2Nb intermetallics is 1.41 ± 0.53 MPam1/2. A simplified scenario of weld formation together with the thermal cycle is proposed to elaborate the way local phase determined the mechanical properties.