Reversible control of magnetization by electric fields without assistance from a subsidiary magnetic field or electric current could help reduce the power consumption in spintronic devices. When increasing temperature above room temperature, FeRh displays an uncommon antiferromagnetic to ferromagnetic phase transition linked to a unit cell volume expansion. Thus, using the strain exerted by an adjacent piezoelectric layer, the relative amount of antiferromagnetic and ferromagnetic regions can be tuned by an electric field applied to the piezoelectric material. Indeed, large variations in the saturation magnetization have been observed when straining FeRh films grown on suitable piezoelectric substrates. In view of its applications, the variations in the remanent magnetization rather than those of the saturation magnetization are the most relevant. Here, we show that in the absence of any bias external magnetic field, permanent and reversible magnetization changes as high as 34% can be induced by an electric field, which remain after this has been zeroed. Bulk and local magnetoelectric characterization reveals that the fundamental reason for the large magnetoelectric response observed at remanence is the expansion (rather than the nucleation) of ferromagnetic nanoregions.