Graphene based van der Waals heterostructure has attracted wide attention recently, especially for graphene/semiconductor Schottky junction. Herein, through delicately designing and engineering the van der Waals heterostructure between graphene and indium phosphide (InP), which has a suitable bandgap of 1.34 eV for solar energy conversion, we have achieved graphene/p-InP solar cells with power conversion efficiency (PCE) of 3.3% under AM 1.5G illumination. The chemical doping or electrical field modulation has been used to tune the Fermi level of graphene, which leads to a PCE of 5.6% for the device under gating effect. Furthermore, the interface recombination rate could be reduced while graphene is doped or gated, as evidenced by transient photoluminescence measurements. Considering the stability of cell performance under illumination and the high resistance to space irradiation damage of InP, graphene/InP heterojunction may be promising for special applications such as space solar cells.