[en] Site dilution of spin-gapped antiferromagnets leads to localized free moments, which can order antiferromagnetically in two and higher dimensions. Here we show how a weak magnetic field drives this order-by-disorder state into a novel disordered-local-moment (DLM) phase, characterized by the formation of local singlets between neighboring moments and by localized moments aligned antiparallel to the field. This disordered phase is characterized by the absence of a gap, as is the case in a Bose glass. The associated field-driven quantum phase transition is consistent with the universality of a superfluid-to-Bose-glass transition. The robustness of the DLM phase and its prominent features, in particular a series of pseudo-plateaus in the magnetization curve, makes it accessible and relevant to experiments