In this paper, the formation control problem of the network of multiple agents is studied in terms of controllability, where the network is of the leader–follower structure with some agents taking leaders role and others being followers interconnected via the neighbor-based rule. It is shown that the controllability of a multi-agent system can be uniquely determined by the topology structure of interconnection graph, for which the investigation comes down to that for a multi-agent system with the interconnection graph being connected. Based on these observations, two kinds of uncontrollable interconnection topologies are characterized, and a necessary and sufficient eigenvector-based condition is presented. Our studies also touch upon the selection of leaders.