We present a detailed analysis of the classical Dicke-Jaynes-Cummings-Gaudin integrable model, which describes a system of $n$ spins coupled to a single harmonic oscillator. We focus on the singularities of the vector-valued moment map whose components are the $n+1$ mutually commuting conserved Hamiltonians. The level sets of the moment map corresponding to singular values may be viewed as degenerate and often singular Arnold-Liouville torii. A particularly interesting example of singularity corresponds to unstable equilibrium points where the rank of the moment map is zero, or singular lines where the rank is one. The corresponding level sets can be described as a reunion of smooth strata of various dimensions. Using the Lax representation, the associated spectral curve and the separated variables, we show how to construct explicitely these level sets. A main difficulty in this task is to select, among possible complex solutions, the physically admissible family for which all the spin components are real. We obtain explicit solutions to this problem in the rank zero and one cases. Remarkably this corresponds exactly to solutions obtained previously by Yuzbashyan and whose geometrical meaning is therefore revealed. These solutions can be described as multi-mode solitons which can live on strata of arbitrary large dimension. In these solitons, the energy initially stored in some excited spins (or atoms) is transferred at finite times to the oscillator mode (photon) and eventually comes back into the spin subsystem. But their multi-mode character is reflected by a large diversity in their shape, which is controlled by the choice of the initial condition on the stratum.