As you've noted, all three of the named substrates are inert, but there's more to a substrate than mineral content. You might have better luck with sand because it's a softer, easier material for roots to grow through than gravel. Also, different substrates will hold fish waste in in different ways: in gravel, the mulm can sink to the bottom of your tank, where as with sand it might get embedded higher up closer to young roots. Sand is also very easy to push plants into. The rough, low-suction texture of gravel can damage the stems and roots of newly purchased plants, giving them a difficult early start.
Under other circumstances there are also advantages to gravel: aggressive fish have a harder time uprooting plants in it, the risk of anaerobic activity can be less, as well as some some special advantages for people who use dirt. My only point is that the nutrient-content aside, there are many ways in which just the shape and mass of substrate can impact growth.