One of the best?
An old privy.
Because of the soft nature of "night soil," (ahem) bottles and such often didn't break. It was also common to move the outhouse after the pit became 'full.' (Do I need to go further?)
When Coors Field was excavated in Denver, a few outhouse pits were uncovered...and descended on by eager diggers. You don't think much about this, unless you love the color and varied shapes of old glassware. Then it's a history-lover's dream.
I just found Dump Diggers, a blog for enthusiasts of this art. There's even a book on the subject ("Digging Up the Past, One Yard At A Time") on these 'toilet treasures.' (Here's another website to whet your interest, as well.)
Husband and I have practiced it some, from putzing around in the backyard of a guy who dug all sorts of things out of old mining towns (and dumped the excess or blemished in his backyard), to an old dump uncovered for a while on the banks of Boulder Creek, while the bike path was being improved on. (I brought home all sorts of cool dish shards from that trip.)
I even tried digging out the site of Ma and Pa's farm's old outhouse by the barn, to the accompaniment of many hoots of laughter. Got the bottom of a pickle glass urn, which I still use for a soap dish. Nothing else, though -- but my grandparents and great-grandparents were dedicated teetotallers. Darn it.