Abstract Here we show that precise Gaia EDR3 proper motions have provided robust estimates of 3D velocities, angular momentum, and total energy for 40 Milky Way dwarfs. The results are statistically robust and are independent of the Milky Way mass profile. Dwarfs do not behave like long-lived satellites of the Milky Way because of their excessively large velocities, angular momenta, and total energies. Comparing them to other MW halo populations, we find that many are at first passage, ≤2 Gyr ago, i.e., more recent than the passage of Sagittarius, ∼4–5 Gyr ago. We suggest that this is in agreement with the stellar populations of all dwarfs, for which we find that a small fraction of young stars cannot be excluded. We also find that dwarf radial velocities contribute too little to their kinetic energy when compared to satellite systems with motions only regulated by gravity, and some other mechanism must be at work such as ram pressure. The latter may have preferentially reduced radial velocities when dwarf progenitors entered the halo until they lost their gas. It could also explain why most dwarfs lie near their pericenter. We also discover a novel large-scale structure perpendicular to the Milky Way disk, which is made by 20% of dwarfs orbiting or counter-orbiting with the Sagittarius dwarf.