Predictive theories of action observation propose that we use our own motor system as a guide for anticipating and understanding other people's actions through the generation of context-based expectations. According to this view, people should be better in predicting and interpreting those actions that are present in their own motor repertoire compared to those that are not. We recorded high-density event-related potentials (ERPs: P300, N400 and Slow Wave, SW) and source estimation in 80 subjects separated by their level of expertise (experts, beginners and naïves) as they observed realistic videos of Tango steps with different degrees of execution correctness. We also performed path analysis to infer causal relationships between ongoing anticipatory brain activity, evoked semantic responses, expertise measures and behavioral performance. We found that anticipatory activity, with sources in a fronto-parieto-occipital network, early discriminated between groups according to their level of expertise. Furthermore, this early activity significantly predicted subsequent semantic integration indexed by semantic responses (N400 and SW, sourced in temporal and motor regions) which also predicted motor expertise. In addition, motor expertise was a good predictor of behavioral performance. Our results show that neural and temporal dynamics underlying contextual action anticipation and comprehension can be interpreted in terms of successive levels of contextual prediction that are significantly modulated by subject's prior experience.