Vast amounts of digital multimedia data are being produced and distributed today, so methods for the efficient and reliable extraction of information from video data are becoming necessary. We present a novel motion segmentation algorithm, which accurately extracts moving objects from a video, and also provides a likelihood map, for each object pixel assignment. The flow is estimated, and accumulated over several frames, to give action masks. Color segmentation clusters regions of similar color in each frame. A novel, likelihood ratio-based method for the statistical comparison of color layers in the regions of activity and the background is presented and compared with an Earth mover's distance-based approach. Our method also gives the likelihood with which each pixel is assigned to a moving object in each frame. Experiments with real sequences illustrate the advantages of our method, namely that it gives overall more reliable results, and also provides the likelihood map for the segmented object.