The chaotic neural network constructed with chaotic neurons exhibits very rich dynamic behaviors and has a nonperiodic associative memory. In the chaotic neural network, however, it is difficult to distinguish the stored patters from others, because the states of output of the network are in chaos. In order to apply the nonperiodic associative memory into information search and pattern identification, etc, it is necessary to control chaos in this chaotic neural network. In this paper, the phase space constraint method focused on the chaotic neural network is proposed. By analyzing the orbital of the network in phase space, we chose a part of states to be disturbed. In this way, the evolutional spaces of the strange attractors are constrained. The computer simulation proves that the chaos in the chaotic neural network can be controlled with above method and the network can converge in one of its stored patterns or their reverses which has the smallest Hamming distance with the initial state of the network. The work clarifies the application prospect of the associative dynamics of the chaotic neural network.