Confidence in a perceptual decision is a judgment about the quality of the sensory evidence. The quality of the evidence depends not only on its strength ('signal') but critically on its reliability ('noise'), but the separate contribution of these quantities to the formation of confidence judgments has not been investigated before in the context of perceptual decisions. We studied subjective confidence reports in a multi-element perceptual task where evidence strength and reliability could be manipulated independently. Our results reveal a confidence paradox: confidence is higher for stimuli of lower reliability that are associated with a lower accuracy. We show that the subjects' overconfidence in trials with unreliable evidence is caused by a reduced sensitivity to stimulus variability. Our results bridge between the investigation of miss-attributions of confidence in behavioral economics and the domain of simple perceptual decisions amenable to neuroscience research.