Simultaneous measurement of X-ray ptychography and fluorescence microscopy allows high-resolution and high-sensitivity observations of the microstructure and trace-element distribution of a sample. In this paper, we propose a method for improving scanning fluorescence X-ray microscopy (SFXM) images, in which the SFXM image is deconvolved via virtual single-pixel imaging using different probe images for each scanning point obtained by X-ray ptychographic reconstruction. Numerical simulations confirmed that this method can increase the spatial resolution while suppressing artifacts caused by probe imprecision, e.g., probe position errors and wavefront changes. The method also worked well in synchrotron radiation experiments to increase the spatial resolution and was applied to the observation of S element maps of ZnS particles.