Autoregulation, toggle switch and relaxation oscillators are important regulatory motifs found in biological gene regulatory networks and interesting results have been reported on theoretical analyses of these regulatory units. However, it is so far unclear how evolution has shaped these motifs based on elementary biochemical reactions. This paper presents a method of designing important dynamics such as bistability and oscillation with these network motifs using an artificial evolutionary algorithm. The evolved dynamics of the network motifs are then verified when the initial states and the parameters of the network motifs are perturbed. It has been found that while it is straightforward to evolve the switching behavior, it is difficult to evolve stable oscillatory dynamics. We show that a higher Hill coefficient will facilitate the generation of undamped oscillation, however, an evolutionary path that can lead to a high Hill coefficient remains an open question for future research.