OBJECTIVE: To evaluate the effect of the time and the sex of victims and perpetrators on the rates of family and community physical violence in a Brazilian municipality over seven years (2008–2014). METHODS: We made a census analysis from non-fatal victims attended in the Forensic Institute of the Scientific Civil Police. The monthly and annual violence rates were calculated based on the population size of the municipality. Time series was evaluated by negative binomial regression models, based on the number of cases with population offset and considering the effect of the sex of victims and perpetrators. RESULTS: A total of 3,324 cases of family and 4,634 cases of community violence were analyzed. There was a significant increase in family violence rates for female victims and male perpetrators. Family violence rates were always higher for female victims than for male and it was always lower for female perpetrators than for male (p < 0.001). There was a lower risk of community violence for male victims after 2013 and a decrease of aggression perpetrated by men over time. Men and women were similarly affected by community violence; however, the perpetrators were more frequently men. CONCLUSIONS: The results indicate a trend of increasing female victims in the family violence, mainly perpetrated by men. The reduction in community violence rates could be the result of policies to reduce crime.