Filament eruptions and hard X-ray (HXR) source motions are commonly observed in solar flares, which provides critical information on the coronal magnetic reconnection. This Letter reports an event on 2005 January 15, in which we found an asymmetric filament eruption and a subsequent coronal mass ejection together with complicated motions of HXR sources during the GOES-class X2.6 flare. The HXR sources initially converge to the magnetic polarity inversion line (PIL), and then move in directions either parallel or perpendicular to the PIL depending on the local field configuration. We distinguish the evolution of the HXR source motion in four phases and associate each of them with distinct regions of coronal magnetic fields as reconstructed using a non-linear force-free field extrapolation. It is found that the magnetic reconnection proceeds along the PIL toward the regions where the overlying field decreases with height more rapidly. It is also found that not only the perpendicular but the parallel motion of the HXR sources correlates well with the HXR lightcurve. These results are discussed in favor of the torus instability as an important factor in the eruptive process. ; Comment: 6 pages, 5 figures, accepted to the Astrophysical Journal Letters