Although robots are starting to enter into our professional and private lives, little is known about the emotional effects they elicit. In line with the Media Equation, humans may react towards robots as they do towards humans, making it all the more important to carefully investigate the preconditions and consequences of contact with robots. Based on assumptions on the socialness of reactions towards robots, we conducted a study that provides further insights into the question of whether humans show emotional reactions towards a robot and whether these reactions differ from those towards a human. To explore emotionality in human–robot interaction we conducted an fMRI study (n = 14). Participants were presented videos showing a human, a robot and an inanimate object, being treated in either an affectionate or in a violent way. Self-reported emotional states and functional imaging data revealed that participants indeed reacted emotionally when seeing the affectionate and violent videos. While no different neural activation patterns emerged for the affectionate interaction towards both, the robot and the human, we found differences in neural activity when comparing only the videos showing abusive behavior indicating that participants experience more emotional distress and show negative empathetic concern for the human in the abuse condition. This was supported by similar findings with regard to participant’s self-reported emotional states.