This study investigated the effect of hydrogel surface roughness on its sliding friction against a solid substrate having modestly adhesive interaction with hydrogels under small normal pressure in water. The friction test was performed between bulk polyacrylamide hydrogels of varied surface roughness and a smooth glass substrate by using a strain-controlled rheometer with parallel-plates geometry. At small pressure (normal strain 1.4-3.6%), the flat surface gel showed a poor reproducibility in friction. In contrast, the gels with a surface roughness of 1-10 μm order showed well reproducible friction behaviors and their frictional stress was larger than that of the flat surface hydrogel. Furthermore, the flat gel showed an elasto-hydrodynamic transition while the rough gels showed a monotonous decrease of friction with velocity. The difference between the flat surface and the rough surface diminished with the increase of the normal pressure. These phenomena are associated with the different contact behaviors of these soft hydrogels in liquid, as revealed by the observation of the interface using a confocal laser microscope.