Observing paired colors with a different hue (in terms of chroma and lightness) engenders pleasantness from such harmonious combinations; however, negative reactions can emerge from disharmonious combinations. Currently, neural mechanisms underlying the esthetic and emotional aspects of color perception remain unknown. The current study reports evidence regarding the neural correlates of color harmony and disharmony. Functional magnetic resonance imaging was used to assess brain regions activated by harmonious or disharmonious color combinations in comparison to other stimuli. Results showed that the left medial orbitofrontal cortex (mOFC) and left amygdala were activated when participants observed harmonious and disharmonious stimuli, respectively. Taken together, these findings suggest that color disharmony may depend on stimulus properties and more automatic neural processes mediated by the amygdala, whereas color harmony is harder to discriminate based on color characteristics and is reflected by the esthetic value represented in the mOFC. This study has a limitation that we could not exclude the effect of preference for color combination, which has a strong positive correlation with color harmony.