AbstractAn important consideration in the design and analysis of randomized trials is the need to account for outcome observations being positively correlated within groups or clusters. Two notable types of designs with this consideration are individually randomized group treatment trials and cluster randomized trials. While sample size methods for testing the average treatment effect are available for both types of designs, methods for detecting treatment effect modification are relatively limited. In this article, we present new sample size formulas for testing treatment effect modification based on either a univariate or multivariate effect modifier in both individually randomized group treatment and cluster randomized trials with a continuous outcome but any types of effect modifier, while accounting for differences across study arms in the outcome variance, outcome intracluster correlation coefficient (ICC) and the cluster size. We consider cases where the effect modifier can be measured at either the individual level or cluster level, and with a univariate effect modifier, our closed‐form sample size expressions provide insights into the optimal allocation of groups or clusters to maximize design efficiency. Overall, our results show that the required sample size for testing treatment effect heterogeneity with an individual‐level effect modifier can be affected by unequal ICCs and variances between arms, and accounting for such between‐arm heterogeneity can lead to more accurate sample size determination. We use simulations to validate our sample size formulas and illustrate their application in the context of two real trials: an individually randomized group treatment trial (the AWARE study) and a cluster randomized trial (the K‐DPP study).