Abstract Aims To investigate the utility of novel deep learning (DL) algorithms in recognizing transposition of the great arteries (TGA) after atrial switch procedure or congenitally corrected TGA (ccTGA) based on routine transthoracic echocardiograms. In addition, the ability of DL algorithms for delineation and segmentation of the systemic ventricle was evaluated. Methods and results In total, 132 patients (92 TGA and atrial switch and 40 with ccTGA; 60% male, age 38.3 ± 12.1 years) and 67 normal controls (57% male, age 48.5 ± 17.9 years) with routine transthoracic examinations were included. Convolutional neural networks were trained to classify patients by underlying diagnosis and a U-Net design was used to automatically segment the systemic ventricle. Convolutional networks were build based on over 100 000 frames of an apical four-chamber or parasternal short-axis view to detect underlying diagnoses. The DL algorithm had an overall accuracy of 98.0% in detecting the correct diagnosis. The U-Net architecture model correctly identified the systemic ventricle in all individuals and achieved a high performance in segmenting the systemic right or left ventricle (Dice metric between 0.79 and 0.88 depending on diagnosis) when compared with human experts. Conclusion Our study demonstrates the potential of machine learning algorithms, trained on routine echocardiographic datasets to detect underlying diagnosis in complex congenital heart disease. Automated delineation of the ventricular area was also feasible. These methods may in future allow for the longitudinal, objective, and automated assessment of ventricular function.