In evolutionary multi-objective optimization, maintaining a good balance between convergence and diversity is particularly crucial to the performance of the evolutionary algorithms. In addition, it becomes increasingly important to incorporate user preferences because it will be less likely to achieve a representative subset of the Pareto optimal solutions using a limited population size as the number of objectives increases. This paper proposes a reference vector guided evolutionary algorithm for many-objective optimization. The reference vectors can not only be used to decompose the original multi-objective optimization problem into a number of single-objective sub-problems, but also to elucidate user preferences to target a preferred subset of the whole Pareto front. In the proposed algorithm, a scalarization approach, termed angle penalized distance, is adopted to balance convergence and diversity of the solutions in the high-dimensional objective space. An adaptation strategy is proposed to dynamically adjust the distribution of the reference vectors according to the scales of the objective functions. Our experimental results on a variety of benchmark test problems show that the proposed algorithm is highly competitive in comparison with five state-of-the-art evolutionary algorithms for many-objective optimization. In addition, we show that reference vectors are effective and cost-efficient for preference articulation, which is particularly desirable for many-objective optimization. Furthermore, a reference vector regeneration strategy is proposed for handling irregular Pareto fronts. Finally, the proposed algorithm is extended for solving constrained many-objective optimization problems.