Power forecasting has a guiding effect on power-aware scheduling strategies to reduce unnecessary power consumption in data centers. Many metrics related to power consumption can be collected in physical servers, such as the status of CPU, memory, and other components. However, most existing methods empirically exploit a small number of metrics to forecast power consumption. To this end, this article uses feature selection based on causality to explore the metrics that strongly influence the power consumption of different tasks. Moreover, we propose a tensor-based dual-stage attention LSTM to forecast the non-linear and non-periodic power consumption. In the proposed model, a multi-way delay embedding transform is utilized to convert the time series into tensors along the temporal direction. The LSTM combines with the tensor technique and the attention mechanism to capture the temporal pattern effectively. In addition, we adopt the cost-sensitive loss function to optimize the specific power forecasting problem in data centers. The experimental results demonstrate that our method can achieve up to 1.4% to 4.3% forecasting accuracy improvement compared with the state-of-the-art models.