A new scheme based on multi-objective hierarchical genetic algorithm (MOHGA) is proposed to extract interpretable rule-based knowledge from data. The approach is derived from the use of multiple objective genetic algorithm (MOGA), where the genes of the chromosome are arranged into control genes and parameter genes. These genes are in a hierarchical form so that the control genes can manipulate the parameter genes in a more effective manner. The effectiveness of this chromosome formulation enables the fuzzy sets and rules to be optimally reduced. Some important concepts about the interpretability are introduced and the fitness function in the MOGA will consider both the accuracy and interpretability of the fuzzy model. In order to remove the redundancy of the rule base proactively, we further apply an interpretability-driven simplification method to newborn individuals. In our approach, we first apply the fuzzy clustering to generate an initial rule-based model. Then the multi-objective hierarchical genetic algorithm and the recursive least square method are used to obtain the optimized fuzzy models. The accuracy and the interpretability of fuzzy models derived by this approach are studied and presented in this paper. We compare our work with other methods reported in the literature on four examples: a synthetic nonlinear dynamic system, a nonlinear static system, the Lorenz system and the Mackey–Glass system. Simulation results show that the proposed approach is effective and practical in knowledge extraction.