We report the surface roughness of a Ti substrate as a critical factor for controlling the degree of the preferred orientation of anatase TiO2 nanotube arrays (NTAs) which are synthesized by anodization and a subsequent annealing process. The degree of the preferred orientation to the (004) plane of the anatase crystal structure has strong dependency on the root-mean-square roughness (Sq) of the initial Ti substrate when the roughness-controlled substrates are anodized in an ethylene glycol-based electrolyte containing ~2 wt% of water. Highly preferred oriented NTAs were obtained from low-Sq (< 10 nm) substrates, which were accompanied by uniform pore distribution and low concentrateon of hydroxyl ions in as-anodized amorphous NTAs. The mechanism of the preferred oriented crystallization of nanometer-scaled tube walls is explained considering the microscopic geometrical uniformity of the oxide barrier and nanopores at the early stage of anodization, which affected the local electric field and thus the insertion of the hydroxyl group into the amorphous TiO2 tube walls.