I find it is somewhat a balance between what is easy and what makes things work better. Dirt can be good for some fussy plants but then it is also prone to being a problem. Depending how much digging, moving plants is done. I find dirt alone to be more trouble than I want as I rarely leave things alone for long and don't like the mess for the small advantage it seems to give me for the plants I use. I do have more of a mix than planned as I do use dirt in pots and that dirt often gets spilled into the tank but with it mixed into the sand, gravel and Flourite that I use most, it gives me little trouble.
But I simply use what I find in the garden and don't work with plants that don't work easy for me to use.