This paper compares the predicted values of the thermal conductivity of a composite made using the equivalent inclusion method (EIM) and the finite element method (FEM) using representative volume elements. The effects of inclusion anisotropy, inclusion orientation distribution, thermal interface conductance, h, and inclusion dimensions have been considered. Both methods predict similar overall behaviour, whereby at high h values, the effective thermal conductivity of the composite is limited by the inclusion anisotropy, while at lower h values, the effect of anisotropy is greatly diminished due to the more dominant effect of limited heat flow across the inclusion/matrix interface. The simulation results are then used to understand why in those cases where it has been possible to produce CNF reinforced Cu matrix composites with a large volume fraction of well dispersed CNFs, the measured thermal properties of the composite have failed to meet the expectations in terms of thermal conductivity, with measured conductivities in the range 200–300 W/m K. The simulation results show that, although degradation of the thermal properties of the CNFs and a poor interfacial thermal conductance are very likely the reasons behind the low conductivities reported, great care should be taken when measuring the thermal conductivity of this new class of materials, to avoid misleading results due to anisotropic effects.