Self-organized criticality (SOC) has attracted considerable interest due to its possible wide ranging implications on a broad range of subjects. However, the experimental observation of SOC using stringent criteria has been difficult and the question of the critical parameters to observe SOC remains open. This is partly due to the fact that there are different criteria applied in order to claim SOC. Here we endeavour to study two aspects of this. First of all, we study the influence of the presence of quenched disorder on the appearance of SOC in the vortex dynamics in niobium by changing the amount of hydrogen impurities. Furthermore, we study whether the roughness properties of the pile surface can be used as a criterion for the appearance of SOC on a par with the observation of finite-size scaling. For this purpose, we compare the roughness and dynamic exponents of the vortex landscape to the avalanche size distribution for different amounts of disorder. The absence of a transition to SOC in the roughness exponent implies that the presence of a rough surface by itself cannot be used as a sufficient criterion for the observation of SOC. A determination of the dynamics of the surface properties, however, shows a transition similar to that of the avalanche properties.