It is a very important issue to quickly retrieve semantic information from a vast multimedia database. In this paper, we propose a statistic-based algorithm to retrieve the videos that contain the requested object motion from video database. In order to speed up our algorithm, we only utilize the local motion embedded in the region-of-interest as the query to retrieve data from MPEG bitstreams. Experimental results demonstrate that our fast video retrieval algorithm is powerful in terms of accuracy and efficiency.