Keep up the water changes until the fertilizer has finished adding nitrate to the water. Lots of water changes.
Having some plants to remove it is good, too, but if there is a light issue, then the plants may not grow fast enough to be very good at removing the nitrate. Better lights are good.
When you use slow release fertilizer that is supposed to be buried, press each one down all the way to the bottom of the tank, and make sure it stays there when you remove your finger or whatever you are using to push it down.
lullafishi, NO3 level makes no difference to the bacteria you are trying to grow when you are cycling a tank. Keep the ammonia and nitrite under 5 ppm, but the nitrate can get WAY high, and not be an issue.