Since we all have different ideas of beautiful, that part might be funky but for rocks of wide variety and cheap prices, I would suggest landscape supply places. Maybe if you narrow down what type you might like, others in your area will have better ideas for free rocks??
Landscape supply places bring in rocks of all sorts from all places so they have a good range available.
I like rocks from lakes and rivers, bring them home and do a bleach water soak to clean them and they are good to go.