Chemical (Carbon) Filtration
If carbon is made to take chemicals out tha water is that taking away from the ferts I give my plants or should I remove the carbon part of the filter?
You can safely remove the carbon, provided you'll commit to larger water changes and to doing them a bit more often. I change out half the water in my tank every week and don't use carbon.
The water changes do a much better job of removing floating particles and toxins in the tank water than chemical medium.
Just a suggestion, though. You're the "Head Waterkeeper".
"Fear not my child, just change the tank water."