The free electron laser (FEL), as the new generation light source, is an attractive tool in scientific frontier research, because of its advantages of full coherence, ultra-short pulse and controllable polarization. Generally, the soft X-ray FEL facilities require a precise measurement of polarization and X-ray energy spectrum. In this paper, based on the soft X-ray FEL user facility under construction at Shanghai, a numerical model in the framework of Geant4 was developed for simulating electron time of flight (e-TOF) based polarimeter and spectrometer. Using this model, the measurement performances' dependences on the critical parameters were studied thoroughly, and an optimal design achieving resolution of 0.5% in polarization fraction and 0.3 eV in X-ray energy spectrum is obtained. ; Comment: 24 pages, 12 figures, X-ray FEL polarimeter and spectrometer for SXFEL user facility