Fracture of metals at the nanoscale and corresponding failure mechanisms have recently attracted considerable interest. However, quantitative in situ fracture experiments of nanoscale metals are rarely reported. Here it is shown that, under uni-axial tensile loading, single crystalline ultrathin gold nanowires may fracture in two modes, displaying distinctively different fracture morphologies and ductility. In situ high resolution transmission electron microscopy (HRTEM) studies suggest that the unexpected brittle-like fracture was closely related to the observed twin structures, which is very different from surface dislocation nucleation/propagation mediated mechanism in ductile fracture mode. Molecular dynamics (MD) simulations further reveal the processes of shear-induced twin formation and damage initiation at the twin structure/free surface interface, confirming the experimentally observed differences in fracture morphology and ductility. Finally, a fracture criterion based on competition between twin formation and surface dislocation nucleation/propagation as a function of misalignment angle is discussed.