A finite element based method is presented for calculating the acoustic radiation force on arbitrarily shaped elastic and fluid particles. Importantly for future applications, this development will permit the modeling of acoustic forces on complex structures such as biological cells, and the interactions between them and other bodies. The model is based on a non-viscous approximation, allowing the results from an efficient, numerical, linear scattering model to provide the basis for the second-order forces. Simulation times are of the order of a few seconds for an axi-symmetric structure. The model is verified against a range of existing analytical solutions (typical accuracy better than 0.1%), including those for cylinders, elastic spheres that are of significant size compared to the acoustic wavelength, and spheroidal particles.