you are set up correctly! plants do not use co2 when lights are off (no photosynthesis going on), so having it run could lead to buildup - enough to maybe be bad for fishies.
on the other side of the coin, neither are your plants making o2 at night (no photosynthesis). depending on how heavy the plant/fishie load is, some people run bubbles at night to ensure enough gas exchange.
i don't have a pH controller, but I believe most people use it the opposite way - to prevent co2 overdose (they have it set to turn OFF once pH gets too LOW). if your pH is getting too high, there is likely another cause and using co2 to lower it is only masking the real problem, as well as potentially gassing your fish.