Planted Tank Enthusiast
Join Date: Mar 2010
Location: Fairfield, CT (USA)
Diatoms, mulm, waste, bacteria, all sorts of stuff (the shrimp do eat the diatoms, yes, but they can't possibly eat every single diatom on every last grain of sand) - all things which are, in moderation, important to the biological stability of an aquarium. Pure white sand is dead sand. You may not like the yellowing look nearly as much from an aesthetic perspective, but it's healthier for the things living in and around it.
But, I'm with Laura Lee on this one - white substrates are a PITA to deal with in the first place and I don't personally think it's worth the effort. Brown, tan, red, and black ones are all much easier to keep looking good.