Significance Structure-based drug design depends on the ability to predict both the three-dimensional structures of candidate molecules bound to their targets and the associated binding affinities. We demonstrate that one can substantially improve the accuracy of these predictions using easily obtained data about completely different molecules that bind to the same target without requiring any target-bound structures of these molecules. The approach we developed to integrate physical and data-driven modeling may find a variety of applications in the rapidly growing field of artificial intelligence for drug discovery.