Grain number per panicle (GNPP), determined mainly by panicle branching, is vital for rice yield. The dissection of the genetic basis underlying GNPP could help to improve rice yield. However, genetic resources, including quantitative trait loci (QTL) or genes for breeders to enhance rice GNPP, are still limited. Here, we conducted the genome-wide association study (GWAS) on the GNPP, primary branch number (PBN), and secondary branch number (SBN) of 468 rice accessions. We detected a total of 18 QTLs, including six for GNPP, six for PBN, and six for SBN, in the whole panel and the indica and japonica subpanels of 468 accessions. More importantly, qPSG1 was a common QTL for GNPP, PBN, and SBN and was demonstrated by chromosome segment substitution lines (CSSLs). Considering gene annotation, expression, and haplotype analysis, seven novel and strong GNPP-related candidate genes were mined from qPSG1. Our results provide clues to elucidate the molecular regulatory network of GNPP. The identified QTLs and candidate genes will contribute to the improvement of GNPP and rice yield via molecular marker-assisted selection (MAS) breeding and genetic engineering techniques.