Photonic structures have been attracting more attention due to their ability to capture, concentrate and propagate optical energy. In this work, we propose a photon-trapping hole-array structure integrated in a nip InAsSb-GaSb heterostructure for the enhancement of the photoresponse in both near- and mid-infrared regions. The proposed symmetrical hole array can increase the photon lifetime inside the absorption layer and reduce reflection without polarization dependence. Significant enhancements in absorption and photoelectric conversion efficiency are demonstrated in dual bands for unpolarized incidence. The enhancement factors of responsivity at room temperature under zero-bias are 1.12 and 1.33 for the near- and mid-infrared, respectively, and they are increased to 1.71 and 1.79 when temperature drops to the thermoelectric cooling temperature of 220 K. Besides, such an integrated hole array also slightly improves working frequency bandwidth and response speed. This work provides a promising way for high-efficiency polarization-independent photoelectric conversion in different electromagnetic wave ranges.