Terra cotta, steve's White, all the usual pottery type clays are suitable. When I was in high school I was heavy into aquariums and sculpture, so I made a ton of weird ornaments, lots of fake (but very convincing!!!) fossils, and such. You should be able to find some local places that will let you work with real "high-fire" clays to make some neat stuff. Most of the usual pottery glazes are safe, but you need to be careful with some of the blues and greens, as they may contain copper that will hurt shrimp and other invertebrates.
There are other materials, but ceramics are usually considered the safest thing that you can easily do yourself.