Walk through an alley or along a small road and pick up discarded materials to work with. Don't be afraid of a little dirt – just be careful not to cut yourself.
As luck would have it, I work two doors down to a paint-your-own-ceramic store, and we share the same back alley/garbage disposal area. Walking around it today, i came across a ripped garbage bag with pieces of unglazed broken ceramic spilling out. It was much more appealing (and clean!) than the other junk scattered around the bins, so i used the pieces to make this star... and left it there for others to enjoy.