The purpose of our study was to investigate whether the whole-brain functional connectivity pattern exhibits disease severity-related alterations in patients with Alzheimer's disease (AD) and mild cognitive impairment (MCI). Resting-state functional magnetic resonance imaging data were acquired in 27 MCI subjects, 35 AD patients, and 27 age- and gender-matched subjects with normal cognition (NC). Interregional functional connectivity was assessed based on a predefined template which parcellated the brain into 90 regions. Altered whole-brain functional connectivity patterns were identified via connectivity comparisons between the AD and NC subjects. Finally, the relationship between functional connectivity strength and cognitive ability according to the mini-mental state examination (MMSE) was evaluated in the MCI and AD groups. Compared with the NC group, the AD group exhibited decreased functional connectivities throughout the brain. The most significantly affected regions included several important nodes of the default mode network and the temporal lobe. Moreover, changes in functional connectivity strength exhibited significant associations with disease severity-related alterations in the AD and MCI groups. The present study provides novel evidence and will facilitate meta-analysis of whole-brain analyses in AD and MCI, which will be critical to better understand the neural basis of AD.