A novel scheme for using modified colloidal crystals as templates for silica polymerization is reported. 3D close-packed crystals of submicrometer latex spheres are assembled on a membrane surface by filtration. To induce silica polymerization, the particles are modified by adsorption of the surfactant hexadecyltrimethylammonium bromide. The colloidal crystals are then infused with a silica solution, which polymerizes in the cavities. In the final stage, the latex particles are removed by calcination, leaving behind porous silica of very low density. Scanning electron microscopy images demonstrate that the product has highly uniform and structured pores, representing a negative replica of the original colloidal crystal. The size of the pores can be controlled by changing the size of the latex used, and we were able to obtain samples with pores ranging from 150 to 1000 nm. Thus the method allows one to obtain structured silica materials of which the pore size, shape, and ordering can be controlled in a wide region that has previously been unattainable.