We aimed to explore the accuracy of physical examination (PE) to detect the synovial and extra-synovial pathologies in psoriatic arthritis (PsA) in comparison to ultrasonography (US). Twenty-nine PsA patients with hand pain were included in the study. A detailed PE of the hands was performed and US scans were performed for the joints, extensor and flexor tendons, and entheses of the second to fifth fingers of both hands. The agreement between PE and US findings was calculated. The strongest agreement for the joints was between “swollen joints” and power Doppler (PD) signals in the metacarpophalangeal (MCP) joints and grey scale synovitis in the proximal interphalangeal (PIP) joints. The agreement of tender entheses on PE and inflammation on US (hypoechogenicity, thickening, and/or PD signals) was poor for both extensor (Kappa = −0.027, Prevalence Adjusted and Bias Adjusted Kappa (PABAK) = 0.344) and flexor compartments (Kappa = 0.039, PABAK = 0.569). Similar to enthesitis, comparison of any PE and US findings showed a poor agreement at the extensor and flexor tendon regions (extensor: Kappa = 0.123, PABAK = 0.448, and flexor: Kappa = 0.171, PABAK = 0.431). Our study showed that there was a poor to fair agreement of PE and US findings of hands. US can add value when determining the source of pain in PsA in the small joints.