No, they won't harm your plants nor will they rob them of nutrient. They are true aquatic and will continue to feed on detritus in the gravel.
However, depends on how much worms you have relative to the tank volume, black worms are animals too and will compete for available O2 while give out CO2. You need to look for signs of O2 deficiency (fish gasping for air at the surface).
Blackworm (aka bloodworm) is larvae form of midge fly. Although non-biting, but having too much of them in the house is a very annoying thing. I would suggest you get a couple Khuli loach for them to dig and search gravel for escaping blackworms.