Characterization of hepatocellular carcinomas (HCCs) and metastatic carcinomas (METs) from B-mode ultrasound presents a daunting challenge for radiologists due to their highly overlapping appearances. The differential diagnosis between HCCs and METs is often carried out by observing the texture of regions inside the lesion and the texture of background liver on which the lesion has evolved. The present study investigates the contribution made by texture patterns of regions inside and outside of the lesions for binary classification between HCC and MET lesions. The study is performed on 51 real ultrasound liver images with 54 malignant lesions, i.e., 27 images with 27 solitary HCCs (13 small HCCs and 14 large HCCs) and 24 images with 27 MET lesions (12 typical cases and 15 atypical cases). A total of 120 within-lesion regions of interest and 54 surrounding lesion regions of interest are cropped from 54 lesions. Subsequently, 112 texture features (56 texture features and 56 texture ratio features) are computed by statistical, spectral, and spatial filtering based texture features extraction methods. A two-step methodology is used for feature set optimization, i.e., feature pruning by removal of nondiscriminatory features followed by feature selection by genetic algorithm–support vector machine (SVM) approach. The SVM classifier is designed based on optimum features. The proposed computer-aided diagnostic system achieved the overall classification accuracy of 91.6 % with sensitivity of 90 % and 93.3 % for HCCs and METs, respectively. The promising results obtained by the proposed system indicate its usefulness to assist radiologists in diagnosing liver malignancies.