Physical Activity (PA) is one of the most important determinants of health. Wearable sensors have great potential for accurate assessment of PA (activity type and Energy Expenditure (EE)) in daily life. In this paper we investigate the benefit of multiple physiological signals (Heart Rate (HR), respiration rate, Galvanic Skin Response (GSR), skin humidity) as well as accelerometer (ACC) data from two locations (wrist - combining ACC, GSR and skin humidity - and chest - combining ACC and HR) on PA type and EE estimation. We implemented single regression, activity recognition and activity-specific EE models on data collected from 16 subjects, while performing a set of PAs, grouped into six clusters (lying, sedentary, dynamic, walking, biking and running). Our results show that combining ACC and physiological signals improves performance for activity recognition (by 2 and 8% for the chest and wrist) and EE (by 36 - chest - and 35% - wrist - for single regression models, and by 18 - chest - and 46% - wrist - for activity-specific models). Physiological signals other than HR showed a coarser relation with level of physical exertion, resulting in being better predictors of activity cluster type and separation between inactivity and activity than EE, due to the weak correlation to EE within an activity cluster.