We present HATNet observations of XO-5b, confirming its planetary nature based on evidence beyond that described in the announcement of Burke et al. (2008), namely, the lack of significant correlation between spectral bisector variations and orbital phase. In addition, using extensive spectroscopic measurements spanning multiple seasons, we investigate the relatively large scatter in the spectral line bisectors. We also examine possible blended stellar configurations (hierarchical triples, chance alignments) that can mimic the planet signals, and we are able to show that none are consistent with the sum of all the data. The analysis of the S activity index shows no significant stellar activity. Our results for the planet parameters are consistent with values in Burke et al. (2008), and we refine both the stellar and planetary parameters using our data. XO-5b orbits a slightly evolved, late G type star with mass M_s = 0.88 +/- 0.03, radius R_s = 1.08 +/- 0.04, and metallicity close to solar. The planetary mass and radius are M_p = 1.059 +/- 0.028 M_Jup and R_p = 1.109 +/- 0.050 R_Jup, respectively, corresponding to a mean density of 0.96 -0.11 +0.14 g/cm^3. The ephemeris for the orbit is P = 4.187757 +/- 0.000011, E= 2454552.67168 +/- 0.00029 (BJD) with transit duration of 0.1307 +/- 0.0013 d. By measuring four individual transit centers, we found no signs for transit timing variations. The planet XO-5b is notable for its anomalously high Safronov number, and has a high surface gravity when compared to other transiting exoplanets with similar period. ; Comment: Accepted for publication in ApJ, 8 pages in emulateapj style