Model-based feedforward-feedback tracking control has been shown as one of the most effective methods for real-time hybrid simulation (RTHS). This approach assumes that the closed-loop servo-hydraulic control system is a linear time-invariant model. However, the closed-loop system is intrinsically nonlinear and time-varying especially when nonlinear experimental components are tested such as magnetorheological dampers (MR damper). In this paper, an adaptive control scheme of a model-based feedforward-feedback control framework is proposed to further improve the tracking performance of the actuator. This adaptive strategy is used to estimate the system parameters for the feedforward controller online during a test. The robust stability of this adaptive controller is provided by introducing Routh's stability criteria and applying the parameter projection algorithm. The proposed control scheme is shown to attain better tracking performance. Finally, RTHS of a nine-story shear building controlled by a full-scale MR damper is carried out to verify the efficacy of the proposed control method. The adaptive feedforward-feedback control scheme is demonstrated effective for structural performance assessment using RTHS.