We report experimental signatures of plasmonic effects due to electron tunneling between adjacent graphene layers. At sub-nanometer separation, such layers can form either a strongly coupled bilayer graphene with a Bernal stacking or a weakly coupled double-layer graphene with a random stacking order. Effects due to interlayer tunneling dominate in the former case but are negligible in the latter. We found through infrared nano-imaging that bilayer graphene supports plasmons with a higher degree of confinement compared to single- and double-layer graphene, a direct consequence of interlayer tunneling. Moreover, we were able to shut off plasmons in bilayer graphene through gating within a wide voltage range. Theoretical modeling indicates that such a plasmon-off region is directly linked to a gapped insulating state of bilayer graphene: yet another implication of interlayer tunneling. Our work uncovers essential plasmonic properties in bilayer graphene and suggests a possibility to achieve novel plasmonic functionalities in graphene few-layers. ; Comment: 27 pages, 11 figures