The thermal conductance of graphene–matrix interfaces plays a key role in controlling the thermal properties of graphene-based nanocomposites. Using atomistic simulations, we found that the interfacial thermal conductance depends strongly on the mode of heat transfer at graphene–matrix interfaces: if heat enters graphene from one side of its basal plane and immediately leaves it through the other side, the corresponding interfacial thermal conductance, Gacross, is large; if heat enters graphene from both sides of its basal plane and leaves it at a position far away on its basal plane, the corresponding interfacial thermal conductance, Gnon-across, is small. For a single-layer graphene immersed in liquid octane, Gacross is ∼150 MW/m2K while Gnon-across is ∼5 MW/m2K. Gacross decreases with increasing multi-layer graphene thickness (i.e., number of layers in graphene) and approaches an asymptotic value of 100 MW/m2K for 7-layer graphenes. Gnon-across increases only marginally as the graphene sheet thickness increases. Such a duality of the interface thermal conductance for different probing methods and its dependence on graphene sheet thickness can be traced ultimately to the unique physical and chemical structure of graphene materials. The ramifications of these results in areas such as the optimal design of graphene-based thermal nanocomposites are discussed.