To overcome the main drawbacks of (μ + λ)-constrained differential evolution ((μ + λ)-CDE) [45], this paper proposes an improved version of (μ + λ)-CDE, named ICDE, to solve constrained optimization problems (COPs). ICDE mainly consists of an improved (μ + λ)-differential evolution (IDE) and a novel archiving-based adaptive tradeoff model (ArATM). Therein, IDE employs several mutation strategies and the binomial crossover of differential evolution (DE) to generate the offspring population. Moreover, a new mutation strategy named “current-to-rand/best/1” is proposed by making use of the current generation number in IDE. Since the population may undergo three situations during the evolution (i.e., the infeasible situation, the semi-feasible situation, and the feasible situation), like (μ + λ)-CDE, ArATM designs one constraint-handling mechanism for each situation. However, unlike (μ + λ)-CDE, in the constraint-handling mechanism of the infeasible situation, the hierarchical nondominated individual selection scheme is utilized, and an individual archiving technique is proposed to maintain the diversity of the population. Furthermore, in the constraint-handling mechanism of the semi-infeasible situation, the feasibility proportion of the combined population consisting of the parent population and the offspring population is used to convert the objective function of each individual. It is noteworthy that ICDE adopts a fixed tolerance value for the equality constraints. In addition, in this paper two criteria are used to compute the degree of constraint violation of each individual in the population, according to the difference among the violations of different constraints. By combining IDE with ArATM, ICDE has the capability to maintain a good balance between the diversity and the convergence of the population during the evolution. The performance of ICDE has been tested on 24 well-known benchmark test functions collected for the special session on constrained real-parameter optimization of the 2006 IEEE Congress on Evolutionary Computation (IEEE CEC2006). The experimental results demonstrate that ICDE not only overcomes the main drawbacks of (μ + λ)-CDE but also obtains very competitive performance compared with other state-of-the-art methods for constrained optimization in the community of constrained evolutionary optimization.