We propose a two-step ligand exchange for the selective end-functionalization of gold nanorods (AuNR) by thiolated cyclodextrin (CD) host molecules. As a result of the complete removal of the precursor capping agent cetyltrimethylammonium bromide (CTAB) by a tetraethylene glycol derivative, competitive binding to the host cavity was prevented, and reversible, light-responsive assembly and disassembly of the AuNR could be induced by host–guest interaction of CD on the nanorods and a photoswitchable arylazopyrazole cross-linker in aqueous solution. The end-to-end assembly of AuNR could be effectively controlled by irradiation with UV and visible light, respectively, over four cycles. By the introduction of AAP, previous disassembly limitations based on the photostationary states of azobenzenes could be solved. The combination photoresponsive interaction and selectively end-functionalized nanoparticles shows significant potential in the reversible self-assembly of inorganic–organic hybrid nanomaterials.