I'd also guess it's mostly tannins from the soil. A certain amount are considered beneficial for a lot of tropical fish, especially South American ones, I believe. Plus, I think Walstad listed more then a couple benefits to planted aquaria from tannins.
Like Stinkypinky12 said, if it bothers you do a water change. I would think that most of the leaching would happen pretty quickly, and after a few water changes or so, it would take longer to turn yellow, or not turn quite as yellow or something. You could also try filtering with carbon, if you have some on hand, I believe that's supposed to be fairly effective at removing tannins.
Using a finer grained cap, like sand or something may help, but I don't think it will really matter much after a month or so.