Abstract Purpose Head and neck squamous cell carcinoma (HNSCC) is one of the most common cancer types globally. Due to the complex anatomy of the region, diagnosis and treatment is challenging. Early diagnosis and treatment are important, because advanced and recurrent HNSCC have a poor prognosis. Robust and precise tools are needed to help diagnose HNSCC reliably in its early stages. The aim of this study was to assess the applicability of a convolutional neural network in detecting and auto-delineating HNSCC from PET-MRI data. Methods 2D U-net models were trained and tested on PET, MRI, PET-MRI and augmented PET-MRI data from 44 patients diagnosed with HNSCC. The scans were taken 12 weeks after chemoradiation therapy with a curative intention. A proportion of the patients had follow-up scans which were included in this study as well, giving a total of 62 PET-MRI scans. The scans yielded a total of 178 PET-MRI slices with cancer. A corresponding number of negative slices were chosen randomly yielding a total of 356 slices. The data was divided into training, validation and test sets (n = 247, n = 43 and n = 66 respectively). Dice score was used to evaluate the segmentation accuracy. In addition, the classification capabilities of the models were assessed. Results When true positive segmentations were considered, the mean Dice scores for the test set were 0.79, 0.84 and 0.87 for PET, PET-MRI and augmented PET-MRI, respectively. Classification accuracies were 0.62, 0.71 and 0.65 for PET, PET-MRI and augmented PET-MRI, respectively. The MRI based model did not yield segmentation results. A statistically significant difference was found between the PET-MRI and PET models (p = 0.008). Conclusion Automatic segmentation of HNSCC from the PET-MRI data with 2D U-nets was shown to give sufficiently accurate segmentations.