We theoretically analyze the properties of metamaterials which have been designed by taking advantage of Babinet's principle. It is shown that the complementary structure exhibits both a complementary spectral response and field distribution of the respective eigenmodes. For complementary split-ring resonators, we show that the spectral resonance features can be explained from two different perspectives. On one hand they can be explained as plasmon polariton resonances in dielectric nanostructures surrounded by metal, on the other hand they can be understood as guided mode resonances with vanishing propagation constant. The physical origin of these modes and differences to the conventional split-ring geometry are discussed.