We present the first X-ray expansion measurement of the SNR Cas A. We compared new ROSAT to archived Einstein HRI images with a separation of 17 years. The remnant is seen to be expanding, on average, at a rate of 0.20 %/yr, which is twice as fast as the expansion of the bright radio ring and two-thirds as fast as the ensemble of optical Fast Moving Knots. This argues that we are observing different hydrodynamical structures in each band, although all of these components are located on the same patchy ring. In addition, significant variations in expansion rate as a function of azimuth around the bright X-ray ring are found. These findings are discussed in the context of two classes of models, involving either an ambient medium with a uniform or monotonically decreasing density away from the center, or a pre-existing circumstellar shell. ; Comment: 9 pages, 2 figures, Submitted to ApJL