The possibility of subcritical behaviour in the geostrophic turbulence regime of rapidly rotating thermally driven convection is explored. In this regime a non-local inverse energy transfer may compete with the more traditional and local direct cascade. We show that, even for control parameters for which no inverse cascade has previously been observed, a subcritical transition towards a large-scale vortex state can occur when the system is initialized with a vortex dipole of finite amplitude. This new example of bistability in a turbulent flow, which may not be specific to rotating convection, opens up new avenues for studying energy transfer in strongly anisotropic three-dimensional flows such as atmospheric or oceanic circulations.