Retrieving information accurately from large image collections is a challenging task due to the presence of complex patterns in visual data. High demands for efficient indexing and retrieval mechanisms is a major motivating factor for the extensive research in this area. Visual features of images are extracted and used for indexing, browsing and retrieval. The human visual system has a sensitivity to colors and edge orientations and hence, these features are very effective for representing a piece of visual information. In this paper, salient image structures corresponding to color and edge orientations are represented as a sparse feature matrix. It combines these features with saliency measures into a unified descriptor. The descriptor is tested on a large dataset. Experimental results verify the capability of the descriptor to represent color images with sufficient discrimination that will enable retrieval systems to perform better as compared to other state-of-the-art methods.