Fatty acid desaturases (FADS) catalyze the formation of unsaturated fatty acids and have been related to insulin sensitivity (IS). FADS activities differ between tissues and are influenced by genetic factors that may impact the link to IS. Genome-wide association studies of δ-5-desaturase (D5D), δ-6-desaturase (D6D) and stearoyl-CoA desaturase-1 (SCD) activities (estimated by product-to-precursor ratios of fatty acids analyzed by gas chromatography) in serum cholesterol esters (n = 1453) and adipose tissue (n = 783, all men) were performed in two Swedish population-based cohorts. Genome-wide significant associated loci were evaluated for associations with IS measured with a hyperinsulinemic euglycemic clamp (n = 554). Variants at the FADS1 were strongly associated with D5D in both cholesterol esters (p = 1.9 × 10−70) and adipose tissue (p = 1.1 × 10−27). Variants in three further loci were associated with D6D in cholesterol esters (FADS2, p = 3.0 × 10−67; PDXDCI, p = 4.8 × 10−8; and near MC4R, p = 3.7 × 10−8) but no associations with D6D in adipose tissue attained genome-wide significance. One locus was associated with SCD in adipose tissue (PKDL1, p = 2.2 × 10−19). Genetic variants near MC4R were associated with IS (p = 3.8 × 10−3). The FADS cluster was the main genetic determinant of estimated FADS activity. However, fatty acid (FA) ratios in adipose tissue and cholesterol esters represent FADS activities in separate tissues and are thus influenced by different genetic factors with potential varying effects on IS.