This research is concerned with the adsorption and desorption of Cu and As(V) on/from different soils and by-products. Both contaminants may reach soils by the spreading of manure/slurries, wastewater, sewage sludge, or pesticides, and also due to pollution caused by mining and industrial activities. Different crop soils were sampled in A Limia (AL) and Sarria (S) (Galicia, NW Spain). Three low-cost by-products were selected to evaluate their bio-adsorbent potential: pine bark, oak ash, and mussel shell. The adsorption/desorption studies were carried out by means of batch-type experiments, adding increasing and individual concentrations of Cu and As(V). The fit of the adsorption data to the Langmuir, Freundlich, and Temkin models was assessed, with good results in some cases, but with high estimation errors in others. Cu retention was higher in soils with high organic matter and/or pH, reaching almost 100%, while the desorption was less than 15%. The As(V) adsorption percentage clearly decreased for higher As doses, especially in S soils, from 60–100% to 10–40%. The As(V) desorption was closely related to soil acidity, being higher for soils with higher pH values (S soils), in which up to 66% of the As(V) previously adsorbed can be desorbed. The three by-products showed high Cu adsorption, especially oak ash, which adsorbed all the Cu added in a rather irreversible manner. Oak ash also adsorbed a high amount of As(V) (>80%) in a rather non-reversible way, while mussel shell adsorbed between 7 and 33% of the added As(V), and pine bark adsorbed less than 12%, with both by-products reaching 35% desorption. Based on the adsorption and desorption data, oak ash performed as an excellent adsorbent for both Cu and As(V), a fact favored by its high pH and the presence of non-crystalline minerals and different oxides and carbonates. Overall, the results of this research can be relevant when designing strategies to prevent Cu and As(V) pollution affecting soils, waterbodies, and plants, and therefore have repercussions on public health and the environment.