The production of runaway massive binaries offers key insights into the evolution of close binary stars and open clusters. The stars HD 14633 and HD 15137 are rare examples of such runaway systems, and in this work we investigate the mechanism by which they were ejected from their parent open cluster, NGC 654. We discuss observational characteristics that can be used to distinguish supernova ejected systems from those ejected by dynamical interactions, and we present the results of a new radio pulsar search of these systems, as well as estimates of their predicted X-ray flux assuming that each binary contains a compact object. Since neither pulsars nor X-ray emission are observed in these systems, we cannot conclude that these binaries contain compact companions. We also consider whether they may have been ejected by dynamical interactions in the dense environment where they formed, and our simulations of four-body interactions suggest that a dynamical origin is possible but unlikely. We recommend further X-ray observations that will conclusively identify whether HD 14633 or HD 15137 contain neutron stars.