We have investigated the effect of partial isovalent anion substitution in Ga1-xMnxAs on electrical transport and ferromagnetism. Substitution of only 2.4% of As by P induces a metal-insulator transition at a constant Mn doping of x=0.046 while the replacement of 0.4% As with N results in the crossover from metal to insulator for x=0.037. This remarkable behavior is consistent with a scenario in which holes located within an impurity band are scattered by alloy disorder in the anion sublattice. The shorter mean free path of holes, which mediate ferromagnetism, reduces the Curie temperature T_{C} from 113 to 60 K (100 to 65 K) upon the introduction of 3.1% P (1% N) into the As sublattice.