Significance People perform tasks in coordination with others in daily life, but the mechanisms are not well understood. Using Granger causality models to examine string quartet dynamics, we demonstrated that musicians assigned as leaders affect other performers more than musicians assigned as followers. These effects were present during performance, when musicians could only hear each other, but were magnified when they could also see each other, indicating that both auditory and visual cues affect nonverbal social interactions. Furthermore, the overall degree of coupling between musicians was positively correlated with ratings of performance success. Thus, we have developed a method for measuring nonverbal interaction in complex situations and have shown that interaction dynamics are affected by social relations and perceptual cues.