Pervasive environments will witness heterogeneous smart embedded devices (e.g. sensors, actuators) integrated into user's living environment (e.g. smart homes and hospitals) and provide a multitude of information that can transparently support user's lifestyle. One promising application resulting from the management and exploitation of this information is the human activity recognition. In this paper we briefly describe our activity recognition architecture and focus on an important management component of this architecture using the concept of object networks. We explore how object networks can integrate various sensor networks and heterogeneous devices into a coherent network through embedded context and role profile and at the same time support distributed context reasoning. The paper also describes the mechanisms used to eliminate and refine context information that is deemed irrelevant due to user behaviour changes over time, by employing the idea of role fitness.