Context: To date, more than 30 planets have been discovered around giant stars, but only one of them has been found to be orbiting within 0.6 AU from the host star, in direct contrast to what is observed for FGK dwarfs. This result suggests that evolved stars destroy/engulf close-in planets during the red giant phase. Aims: We are conducting a radial velocity survey of 164 bright G and K giant stars in the southern hemisphere with the aim of studying the effect of the host star evolution on the inner structure of planetary systems. In this paper we present the spectroscopic atmospheric parameters (\Teff, \logg, $ξ$, [Fe/H]) and the physical properties (mass, radius, evolutionary status) of the program stars. In addition, rotational velocities for all of our targets were derived. Methods: We used high resolution and high S/N spectra to measure the equivalent widths of many Fe{\sc\,i} and Fe{\sc\,ii} lines, which were used to derive the atmospheric parameters by imposing local thermodynamic and ionization equilibrium. The effective temperatures and metallicities were used, along with stellar evolutionary tracks to determine the physical properties and evolutionary status of each star. Results: We found that our targets are on average metal rich and they have masses between $∼$\,1.0\,M$_⊙$ and 3.5\,M$_⊙$. In addition, we found that 122 of our targets are ascending the RGB, while 42 of them are on the HB phase. ; Comment: Accepted for publication in A&A