The intraflagellar transport (IFT) complex is an integral component of the cilium, a quintessential organelle of the eukaryotic cell. The IFT system consists of three subcomplexes [i.e., intraflagellar transport (IFT)-A, IFT-B, and the BBSome], which together transport proteins and other molecules along the cilium. IFT dysfunction results in diseases collectively called ciliopathies. It has been proposed that the IFT complexes originated from vesicle coats similar to coat protein complex (COP) I, COPII, and clathrin. Here we provide phylogenetic evidence for common ancestry of IFT subunits and alpha, beta', and epsilon subunits of COPI, and trace the origins of the IFT-A, IFT-B, and the BBSome subcomplexes. We find that IFT-A and the BBSome likely arose from an IFT-B-like complex by intracomplex subunit duplication. The distribution of IFT proteins across eukaryotes identifies the BBSome as a frequently lost, modular component of the IFT. Significantly, loss of the BBSome from a taxon is a frequent precursor to complete cilium loss in related taxa. Given the inferred late origin of the BBSome in cilium evolution and its frequent loss, the IFT complex behaves as a "last-in, first-out" system. The protocoatomer origin of the IFT complex corroborates involvement of IFT components in vesicle transport. Expansion of IFT subunits by duplication and their subsequent independent loss supports the idea of modularity and structural independence of the IFT subcomplexes.