In vitro non-cellular permeability models such as the parallel artificial membrane permeability assay (PAMPA) are widely applied tools for early-phase drug candidate screening. In addition to the commonly used porcine brain polar lipid extract for modeling the blood–brain barrier’s permeability, the total and polar fractions of bovine heart and liver lipid extracts were investigated in the PAMPA model by measuring the permeability of 32 diverse drugs. The zeta potential of the lipid extracts and the net charge of their glycerophospholipid components were also determined. Physicochemical parameters of the 32 compounds were calculated using three independent forms of software (Marvin Sketch, RDKit, and ACD/Percepta). The relationship between the lipid-specific permeabilities and the physicochemical descriptors of the compounds was investigated using linear correlation, Spearman correlation, and PCA analysis. While the results showed only subtle differences between total and polar lipids, permeability through liver lipids highly differed from that of the heart or brain lipid-based models. Correlations between the in silico descriptors (e.g., number of amide bonds, heteroatoms, and aromatic heterocycles, accessible surface area, and H-bond acceptor–donor balance) of drug molecules and permeability values were also found, which provides support for understanding tissue-specific permeability.