We present cosmological constraints from the Dark Energy Survey (DES) using a combined analysis of angular clustering of red galaxies and their cross-correlation with weak gravitational lensing of background galaxies. We use a 139 square degree contiguous patch of DES data from the Science Verification (SV) period of observations. Using large scale measurements, we constrain the matter density of the Universe as Omega_m = 0.31 +/- 0.09 and the clustering amplitude of the matter power spectrum as sigma_8 = 0.74 +/- 0.13 after marginalizing over seven nuisance parameters and three additional cosmological parameters. This translates into S_8 = sigma_8(Omega_m/0.3)^{0.16} = 0.74 +/- 0.12 for our fiducial lens redshift bin at 0.35