Computational fluid dynamics (CFD) simulations have been extensively used in many aerodynamic design optimization problems, such as wing and turbine blade shape design optimization. However, it normally takes very long time to solve such optimization problems due to the heavy computation load involved in CFD simulations, where a number of differential equations are to be solved. Some efforts have been seen using feedforward neural networks to approximate CFD models. However, feedforward neural network models cannot capture well the dynamics of the differential equations. Thus, training data from a large number of different designs are needed to train feedforward neural network models to achieve reliable generalization. In this work, a technique using differential recurrent neural networks has been proposed to predict the performance of candidate designs before the CFD simulation is fully converged. Compared to existing methods based on feedforward neural networks, this approach does not need a large number of previous designs. Case studies show that the proposed method is very promising.