Ubiquitin specific proteases (USPs) are the largest family of deubiquitinating enzymes with approximately 56 members in humans. USPs regulate a wide variety of cellular processes by their ability to remove (poly)ubiquitin from target proteins. Their enzymatic activity is encoded in a common catalytic core of approximately 350 amino acids, however many USPs show significantly larger catalytic domains. Here we have analysed human and yeast USP domains, combining bioinformatics with structural information. We reveal that all USP domains can be divided into six conserved boxes, and we map the conserved boxes onto the USP domain core structure. The boxes are interspersed by insertions, some of which as large as the catalytic core. The two most common insertion points place inserts near the distal ubiquitin binding site, and in many cases ubiquitin binding domains or ubiquitin-like folds are found in these insertions, potentially directly affecting catalytic function. Other inserted sequences are unstructured, and removal of these might aid future structural and functional analysis. Yeast USP domains have a different pattern of inserted sequences, suggesting that the insertions are hotspots for evolutionary diversity to expand USP functionality.