To date, the interplay betwexen neurophysiological and individual difference factors in altruistic punishment has been little understood. To examine this issue, 45 individuals participated in a Dictator Game with punishment option while the feedback-related negativity (FRN) was derived from the electroencephalogram (EEG). Unlike previous EEG studies on the Dictator Game, we introduced a third party condition to study the effect of fairness norm violations in addition to employing a first person perspective. For the first time, we also examined the role of individual differences, specifically fairness concerns, positive/negative affectivity, and altruism/empathy as well as recipients' financial situation during altruistic punishment. The main results show that FRN amplitudes were more pronounced for unfair than for fair assignments in both the first person and third party perspectives. These findings suggest that FRN amplitudes are sensitive to fairness norm violations and play a crucial role in the recipients' evaluation of dictator assignments. With respect to individual difference factors, recipients' current financial situation affected the FRN fairness effect in the first person perspective, indicating that when being directly affected by the assignments, more affluent participants experienced stronger violations of expectations in altruistic punishment decisions. Regarding individual differences in trait empathy, in the third party condition FRN amplitudes were more pronounced for those who scored lower in empathy. This may suggest empathy as another motive in third party punishment. Independent of the perspective taken, higher positive affect was associated with more punishment behavior, suggesting that positive emotions may play an important role in restoring violated fairness norms.