Folks should run a stop test to validate that their regulator is actually regulating accurately. Otherwise, when their solenoid closes, pressure could build up on the LP side (and then they'll dump their cylinder via the relief valve).
I think I've already mention this problem in this or some other related threads, it is internal poppet valve leak, so the LP pressure slowly climb even the regulator at off position, and the build up pressure at the LP side will eventually damage something, push through the solenoid or the relieve valve if there is one.
perform a blow test, may or may not fix the problem.
If the problem persists after the blow test, it is broken, and such problem are commonly seen on commercially available regulators.
I will add this to the steps, thanks.