We report results of a muon spin rotation and relaxation ($μ$SR) study of dilute Pd$_{1-x}$Ni$_x$ alloys, with emphasis on Ni concentrations $x =$ 0.0243 and 0.025. These are close to the critical value $x_\mathrm{cr}$ for the onset of ferromagnetic long-range order (LRO), which is a candidate for a quantum critical point. The 2.43 and 2.5 at.% Ni alloys exhibit similar $μ$SR properties. Both samples are fully magnetic, with average muon local fields $〈 B^\mathrm{loc}〉 =$ 2.0 and 3.8 mT and Curie temperatures $T_C =$ 1.0 and 2.03 K for 2.43 and 2.5 at.% Ni, respectively, at $T = 0$. The temperature dependence of $〈 B^\mathrm{loc}〉$ suggests ordering of Ni spin clusters rather than isolated spins. Just above $T_C$ a two-phase region is found with separate volume fractions of quasistatic short-range order (SRO) and paramagnetism. The SRO fraction decreases to zero with increasing temperature a few kelvin above $T_C$. This mixture of SRO and paramagnetism is consistent with the notion of an inhomogeneous alloy with Ni clustering. The measured values of $T_C$ extrapolate to $x_\mathrm{cr}$ = 0.0236 $±$ 0.0027. The dynamic muon spin relaxation in the vicinity of $T_C$ differs for the two samples: a relaxation-rate maximum at $T_C$ is observed for $x$ = 0.0243, reminiscent of critical slowing down, whereas for $x =$ 0.025 no dynamic relaxation is observed within the $μ$SR time window. The data suggest a mean-field-like transition in this alloy. ; Comment: 15 pages, 15 figures, to be published in Phys. Rev. B