Cosmological $N$-body simulations show that Milky Way-sized galaxies harbor a population of unmerged dark matter subhalos. These subhalos could shine in gamma-rays and be eventually detected in gamma-ray surveys as unidentified sources. We performed a thorough selection among unidentified Fermi-LAT Objects (UFOs) to identify them as possible TeV-scale dark matter subhalo candidates. We search for very-high-energy (E $\gtrsim$ 100 GeV) gamma-ray emissions using H.E.S.S. observations towards four selected UFOs. Since no significant very-high-energy gamma-ray emission is detected in any dataset of the four observed UFOs nor in the combined UFO dataset, strong constraints are derived on the product of the velocity-weighted annihilation cross section $〈 σv 〉$ by the $J$-factor for the dark matter models. The 95% C.L. observed upper limits derived from combined H.E.S.S. observations reach $〈 σv 〉 J$ values of 3.7$\times$10$^{-5}$ and 8.1$\times$10$^{-6}$ GeV$^2$cm$^{-2}$s$^{-1}$ in the $W^+W^-$ and $τ^+τ^-$ channels, respectively, for a 1 TeV dark matter mass. Focusing on thermal WIMPs, the H.E.S.S. constraints restrict the $J$-factors to lie in the range 6.1$\times$10$^{19}$ - 2.0$\times$10$^{21}$ GeV$^2$cm$^{-5}$, and the masses to lie between 0.2 and 6 TeV in the $W^+W^-$ channel. For the $τ^+τ^-$ channel, the $J$-factors lie in the range 7.0$\times$10$^{19}$ - 7.1$\times$10$^{20}$ GeV$^2$cm$^{-5}$ and the masses lie between 0.2 and 0.5 TeV. Assuming model-dependent predictions from cosmological N-body simulations on the $J$-factor distribution for Milky Way-sized galaxies, the dark matter models with masses greater than 0.3 TeV for the UFO emissions can be ruled out at high confidence level.