You'll have better luck with finer-grained gravel than thicker stuff. But by "gravel", do you mean actual rock or epoxy-coated neon stuff they sell at big chain pet stores (which you will assuredly see referred to as "clown puke")?
As for adding dirt, yes, you can add dirt (look up "Walstad tank"), though Miracle Gro Organic Potting Mix might be a better bet as a bottom layer. Mineralized TopSoil (check the MTS sticky in the substrate forum) would be ideal, but that takes time and preparation. No matter what you do, though, it will be difficult to add a bottom layer without completely removing whatever you have now, first. I'm not sure, from your post, if you'd be starting from scratch or not.