The antibacterial properties of cellulose acetate/silver nanoparticles (AgNP) ultrafiltration membranes were correlated with their integral asymmetric porous structures, emphasizing the distinct features of each side of the membranes, that is, the active and porous layers surfaces. Composite membranes were prepared from casting solutions incorporating polyvinylpyrrolidone-covered AgNP using the phase inversion technique. The variation of the ratio acetone/formamide and the AgNP content resulted in a wide range of asymmetric porous structures with different hydraulic permeabilities. Comprehensive studies assessing the antibacterial activity against Escherichia coli (cell death and growth inhibition of bacteria in water) were performed on both membrane surfaces and in E. coli suspensions. The results were correlated with the surface chemical composition assessed by XPS. The silver-free membranes presented a generalized growth of E. coli, which is in contrast with the inhibition patterns displayed by the membranes containing AgNP. For the surface bactericide test, the growth inhibition depends on the accessibility of E. coli to the silver present in the membrane; as the XPS results show, the more permeable membranes (CA30 and CA34 series) have higher silver signal detected by XPS, which is correlated with a higher growth inhibition. On the other hand, the inhibition action is independent of the membrane porous structure when the membrane is deeply immersed in an E. coli inoculated suspension, presenting almost complete growth inhibition.