Abstract In the development of silicon photonics, the continued downsizing of photonic integrated circuits will further increase the integration density, which augments the functionality of photonic chips. Compared with the traditional design method, inverse design presents a novel approach for achieving compact photonic devices. However, achieving compact, reconfigurable photonic devices with the inverse design that employs the traditional modulation method exemplified by the thermo-optic effect poses a significant challenge due to the weak modulation capability. Low-loss phase change materials (PCMs) exemplified by Sb₂Se₃ are a promising candidate for solving this problem benefiting from their high refractive index contrast. In this work, we first developed a robust inverse design method to realize reconfigurable silicon and phase-change materials hybrid photonic devices including mode converter and optical switch. The mode converter exhibits a broadband operation of >100 nm. The optical switch shows an extinction ratio of >25 dB and a multilevel switching of 41 (>5 bits) by simply changing the crystallinity of Sb₂Se₃. Here, we experimentally demonstrated a Sb₂Se₃/Si hybrid integrated optical switch for the first time, wherein routing can be switched by the phase transition of the whole Sb₂Se₃. Our work provides an effective solution for the design of photonic devices that is insensitive to fabrication errors, thereby paving the way for high integration density in future photonic chips.