I was wondering if my plants will need fertilizers, how much CO2, and what amount of water changes being under 2.9watts per gallon of light?
In general, yes plants need fertilizer and yes they need CO2 to grow. You mention later on that you have a soil substrate covered with sand - this gives you a few advantages. Soil and peat provide an organic source of carbon, but it gets depleted over time and eventually to get your plants to grow well again you'll have to replace the soil (about every year or a bit longer). It can also provide many, if not all, of the nutrients that plants need to grow. Diana Walsted wrote a fantastic book on this low-tech approach - "The Ecology of the Planted Aquarium". Not many pictures, but a fantastic read!
Water changes are subjective, depending on many things. I do mine every two weeks. I've seen tanks go completely without - though this is rare. Monthly is the standard I think.
I currently have root tabs for my melon sword.
Right on - they are heavy root feeders.
I use three DIY 2liter CO2 bottles on my 55gal tank. Do I need to increase this? Is it possible to overdose CO2? If so, what happens? I was also thinking about getting the Hagen Plant Grow Natural System with CO2. Is this system worth it compared to DIY?
If you're dissolving it well then three should be enough. I admire you - keeping 3 DIY bottles going is more trouble than I want to deal with! But yes, if dissolved well, it should be enough. Do you have a reactor? Venturi-style reactors work very well and aren't too tough to make. It IS possible to OD on CO2 from the fishes perspective (but not for the plants). If you ever see your fish gasping near the surface, just do a good water change and run an airstone for a bit and they'll generally bounce back. If you're already doing DIY CO2, there's really no reason to get a Hagen system, except maybe for the ladder. It is EXACTLY the same thing as DIY CO2, except in a smaller container.
Right now I'm having problems with blue-green algae and hair algae mostly.
Many times BGA will pop up when your nitrates bottom out. If you're not fertilizing, well, then I guess that makes sense.
I have a relatively low plant mass. I will be getting more plants today. I want to use seachem flourish excel but I can't find it in my lfs. I don't want to buy it online because it cost $14 a bottle (for a small bottle), including the shipping. I just can't bring myself to pay that for a small bottle of it. Should I use regular flourish instead? What are the differences between the two?
More plants = better! And Flourish is a very good trace element fertilizer. It'll help plant growth, but it lacks a lot of the macro nutrients (NO3, PO4, K). Seachem sells the macros separate though, and so does Greg Watson. Macros are what your plants need in large amounts, so some source - be it substrate, fish, ferts, etc. of these is necessary for healthy growth.
Seachem EXCEL is a whole 'nother beast. I think it is the best thing since sliced bread. It is completely unique to Seachem - think of it as basically liquid CO2. The chemical makeup is similar to a sugar in the intermediate stage between photosynthesis and consumption, carbon in a usable form. Not only this, but Excel is a mild algaecide. They can't advertise it as such, but I guarantee that once you start using it you'll see SIGNIFICANTLY less algae. Many people use it as a spot treatment for certain types (BBA especially). I buy it by the 4L jug, and can't say enough good things about Excel. I paid... $45? I believe that was including shipping, for a 4L jug from Big Al's. If you'd like to try some Excel, go ahead and PM me your address and I'll send you a little bottle from my big jug to try. You'll love it.
You didn't mention what size the tank was, but yes, I do believe it is possible!