A number of application areas such as biomedical engineering require solving an underdetermined linear inverse problem. In such a case, it is necessary to make assumptions on the sources to restore identifiability. This problem is encountered in brain-source imaging when identifying the source signals from noisy electroencephalographic or magnetoencephalographic measurements. This inverse problem has been widely studied during recent decades, giving rise to an impressive number of methods using different priors. Nevertheless, a thorough study of the latter, including especially sparse and tensor-based approaches, is still missing. In this article, we propose 1) a taxonomy of the algorithms based on methodological considerations; 2) a discussion of the identifiability and convergence properties, advantages, drawbacks, and application domains of various techniques; and 3) an illustration of the performance of seven selected methods on identical data sets. Directions for future research in the area of biomedical imaging are eventually provided.