I'm a big fan of greatest hits/essentials type collections for building a library, especially when they're two cd sets. You get a bit more for less money and you get all the best tunes. I had my car stolen just over a year ago and I lost tons of music. Since then I've been rebuilding my collection and in some cases getting compilations instead of replacing everything.

Here's my list of cds I would die without:
The Who - Tommy; Then & Now (1964 - 2004)
The Clash - The Essential Clash
Bob Dylan - The Essential Bob Dylan
Queen - Live at Wembley Stadium
Alison Krauss & Union Station - anything and everything
A.F.I. - Sing the Sorrow

Others that I just love:
Kim Richey - Rise
David Bowie - Best of Bowie (2-disc set)
Badly Drawn Boy - The Hour of the Bewilderbeast
Rancid - And Out Come the Wolves
Indigo Girls - 1200 Curfews