When I've made backgrounds/paludariums before, i just straight up mixed sand into a cup of epoxy, and then stuck it on the styrofoam. I thought it looked sufficiently rock-like, but it is somewhat glossy from the epoxy. On the above-water portions, I would sprinkly sand over the epoxy, and then brush off/vacuum off the excess once the epoxy cured. I didn't bother with the underwater portions, because it wasn't really noticeable.
I've heard of people using chinchilla dust for this, but I haven't tried that myself, I just used whatever sand I was mixing in with it to begin with.
If you were really concerned about structural strength, you could use fiberglass mats, or even just old rags or paper or something, but just the epoxy/sand mix is fairly strong itself.