The role of membrane pore size and solution chemistry in ultrafiltration (UF) membrane fouling by colloidal natural organic matter (NOM) has been investigated. Fouling experiments were conducted with two laboratory-made cellulose acetate UF membranes, with estimated pore sizes of 2 and 10 nm, under identical hydrodynamic conditions. Flux decline with colloidal NOM was independent of pH and increased in the presence of calcium. The membrane pore size has been found to influence membrane flux decline. Permeate flux for the more permeable membrane (10 nm pore size) decreased faster than the flux of the tightest, less permeable membrane (2 nm pore size). The adhesion forces between the foulant (colloidal NOM) and the clean membrane, and between the foulant in bulk solution and foulant in the fouling layer, were determined by atomic force microscopy (AFM). Force measurements confirmed the fouling trends with calcium ions. For a given solution chemistry, foulant–foulant adhesion was identical for both membranes, indicating that the more pronounced flux decline observed for the largest pore-size membrane was not related to differences in intermolecular chemical interactions among NOM molecules. At the earlier stage of filtration, the dominant fouling mechanism was pore blocking for both membranes. For longer filtration times, there was a transition in the fouling mechanism from pore blocking to cake formation. This transition occurred first for the more permeable membrane, indicating that colloidal NOM penetrates more readily into the pores of this membrane and that the relative size of colloidal NOM and the membrane pore influences the rate of pore blocking. The results further suggest that the structure of the fouling layer is dependent on the operating pressure. At high pressure, the compactness and specific resistance of the colloidal NOM layer substantially increase, and thus, significantly affect the extent of flux decline.