Task switching research has revealed that task changes lead to a performance switch cost. The present study focuses on the organization of task components in the task set. Three different views of task set organization have been distinguished and evidence in favor of each of these has been reported in the literature. In four experiments, we orthogonally varied the categorization task (magnitude and parity) and the stimulus dimension on which the categorization was to be made. Experiments 1, 2, and 4 used Stroop-like number stimuli, whereas Experiment 3 used global-local stimuli to define the stimulus dimension. In Experiments 2-4, the cue-stimulus interval was also varied. The findings showed that a change of any component resulted in a cost, without any reliable difference in the size of these costs. These results are consistent with the flat view on task-set organization, which assumes that the task set binds all elements in an unstructured representation, which is completely reconfigured each time a change to the task set is required. The implications of these findings are discussed in relation to other findings and the different views on task-set organization.