Abstract Cardiac diffusion tensor imaging (DTI) is a noninvasive method for measuring the microstructure of the myocardium. However, its long scan time significantly hinders its wide application. In this study, we developed a deep learning framework to obtain high-quality DTI parameter maps from six diffusion-weighted images (DWIs) by combining deep-learning-based image generation and tensor fitting, and named the new framework FG-Net. In contrast to frameworks explored in previous deep-learning-based fast DTI studies, FG-Net generates inter-directional DWIs from six input DWIs to supplement the loss information and improve estimation accuracy for DTI parameters. FG-Net was evaluated using two datasets of ex vivo human hearts. The results showed that FG-Net can generate fractional anisotropy, mean diffusivity maps, and helix angle maps from only six raw DWIs, with a quantification error of less than 5%. FG-Net outperformed conventional tensor fitting and black-box network fitting in both qualitative and quantitative metrics. We also demonstrated that the proposed FG-Net can achieve highly accurate fractional anisotropy and helix angle maps in DWIs with different b-values.