Self-assembled growth of Ge quantum dot lattices in oxide matrices prepared by the quite simple magnetron sputtering deposition method allows the preparation of a variety of structures tunable by their shape, size and arrangement. The driving mechanism for the self-assembly was attributed to the surface morphology features originating from the quantum dots' growth. Here it is shown specifically that the matrix type is another critical factor that enables the control of the self-assembly process and the tuning of the ordering type and degree of regularity of quantum dot systems. The effectiveness of the matrix factor is demonstrated through the analysis of quantum dot arrangements in amorphous silica, alumina and mullite matrices. Using the same deposition conditions, different ordering types and degrees of disorder were found in the quantum dot systems based on different matrices. The matrix factor is shown to be driven by different matrix tendencies to smooth the surface during the growth of the films. The obtained results are relevant for understanding and tailoring of the self-assembled growth of quantum dot lattices in amorphous systems.