We show that non-Abelian lattice gauge fields can be simulated with a single component ultra-cold atomic gas in an optical lattice potential. An optical lattice can be viewed as a Bravais lattice with a $N$-point basis. An atom located at different points of the basis can be considered as a {\it particle} in different internal states. The appropriate engineering of tunneling amplitudes of atoms in an optical lattice allows one to realize U$(N)$ gauge potentials and control a mass of {\it particles} that experience such non-Abelian gauge fields. We provide and analyze a concrete example of an optical lattice configuration that allows for simulation of a static U(2) gauge model with a constant Wilson loop and an adjustable mass of {\it particles}. In particular, we observe that the non-zero mass creates large conductive gaps in the energy spectrum, which could be important in the experimental detection of the transverse Hall conductivity. ; Comment: 6 pages, 5 figures, version accepted for publication in EPL