The main coffee diterpenes cafestol, kahweol, and 16-O-methylcafestol, present in the bean lipid fraction, are mostly esterified with fatty acids. They are believed to induce dyslipidaemia and hypercholesterolemia when taken with certain types of coffee brews. The study of their binding to serum albumins could help explain their interactions with biologically active xenobiotics. We investigated the interactions occurring between cafestol and 16-O-methylcafestol palmitates with Bovine Serum Albumin (BSA), Human Serum Albumin (HSA), and Fatty Free Human Serum Albumin (ffHSA) by means of circular dichroism and fluorimetry. Circular Dichroism (CD) revealed a slight change (up to 3%) in the secondary structure of fatty-free human albumin in the presence of the diterpene esters, suggesting that the aliphatic chain of the palmitate partly occupies one of the fatty acid sites of the protein. A warfarin displacement experiment was performed to identify the binding site, which is probably close but not coincident with Sudlow site I, as the affinity for warfarin is enhanced. Fluorescence quenching titrations revealed a complex behaviour, with Stern–Volmer constants in the order of 103–104 Lmol−1. A model of the HSA-warfarin-cafestol palmitate complex was obtained by docking, and the most favourable solution was found with the terpene palmitate chain inside the FA4 fatty acid site and the cafestol moiety fronting warfarin at the interface with site I.