Topotactic transformations between related crystal structures are a powerful emerging route for the synthesis of novel quantum materials. Whereas most such "soft chemistry" experiments have been carried out on polycrystalline powders or thin films, the topotactic modification of single crystals -- the gold standard for physical property measurements on quantum materials -- has proven remarkably difficult. Here, we report the topotactic reduction of La$_{1-x}$Ca$_{x}$NiO$_3$ single-crystals to La$_{1-x}$Ca$_{x}$NiO$_{2+δ}$ using CaH$_2$ as the reducing agent. The transformation from the three-dimensional perovskite to the quasi-two-dimensional infinite-layer phase was thoroughly characterized by X-ray diffraction, electron microscopy, Raman spectroscopy, magnetometry, and electrical transport measurements. Our work demonstrates that the infinite-layer structure can be realized as a bulk phase in crystals with micrometer-sized single-domains. The electronic properties of these specimens resemble those of epitaxial thin films rather than powders with similar compositions.