It depends on what you're interested in, because TP writes Discworld books about several topics: wizards, witches, City Watch (or is it guardhouse!?), Death, a few miscellaneous things, and tell me if I've forgotten something. I personally like witches and Death the most. From there, you probably start with the first book about any of them. 

Sorry if I got something confused, which I probably did. And I probably left a lot out. I'll leave this to TNO to continue.  

So where do I start? Wikipedia, for once, isn't helpful.

Depends what you like, really. Unless otherwise noted I'd suggest starting with the first book of each subseries and going from there.

Subseries are as follows (keeping in mind that there will almost always be some overlap): 

The City Watch books:

Guards! Guards!, Men at Arms, Feet of Clay, Jingo, The Fifth Elephant, Night Watch,* Thud! and (kinda sorta) Snuff (less "Watch" and more "Vimes" in this case).

The Rincewind/Wizards/UU books:

The Color of Magic,† The Light Fantastic, Sourcery, Eric, Interesting Times, The Last Continent, The Last Hero Unseen Academicals (UA being much more heavily focused on the UU and the wizards than on Rincewind) + Science novels

The Lancre Witches books:

Equal Rites, Wyrd Sisters, Witches Abroad, Lords and Ladies, Maskerade, Carpe Jugulum

The Tiffany Aching books are an (ostensibly) YA-focused subset of the LW books:

The Wee Free Men, A Hat Full of Sky, Wintersmith, I Shall Wear Midnight

The Death books (Mort or Hogfather are the ones I would reccommend reading first. The thing with Susan will be slightly confusing, esp. if you start with Hogfather instead of Mort, but Reaper Man doesn't fit into the continuity very well since it's very isolated and Soul Music is really confusing if you aren't familiar with the series):

Mort, Reaper Man Soul Music, Hogfather, Thief of Time*

The Moist von Lipwig books (reccommend not to read until after Thud!, for reasons of understanding characterization better):

Going Postal, Making Money

Unsorted:

Pyramids, Moving Pictures (wizard subplot), Small Gods (reccommended to read before Thief of Time, since it introduces the History Monks), The Truth, Monstrous Regiment, The Amazing Maurice and His Educated Rodents (YA)

*Night Watch takes place during the last part of Thief of Time, which is important to bear in mind while reading either, but it is not necessary to read Thief of Time to understand Night Watch or vice versa. 

†Speaking in terms of release date, this is the first book of the Discworld series as a whole. 

Now, let's see, a reccommended reading order... (My head!canon) in-universe time line goes something like this, again divided by subseries for convenience:

Ancient Civilizations

Pyramids --> Small Gods

Early Century of the Fruitbat

The Color of Magic/The Light Fantastic-->Sourcery-->Eric

Equal Rights-->Wyrd Sisters

Mort-->Reaper Man 

Guards! Guards!-->Men at Arms 

Industrial Revolution

Interesting Times/The Last Continent-->The Science of Discworld I, II, III-->The Last Hero-->Unseen Academicals 

Witches Abroad-->Lords and Ladies-->Maskerade-->Carpe Jugulum-->The Wee Free Men-->A Hat Full of Sky-->I Shall Wear Midnight

Soul Music-->Hogfather-->Thief of Time

Feet of Clay-->Jingo-->The Fifth Elephant-->Night Watch-->Thud!-->Snuff

Moving Pictures----------------->The Truth-->Monstrous Regiment-->Going Postal-->Making Money

(Time... is a little funny on the Discworld. There are no continuity errors, just alternate histories!)

Good starting books are Wyrd Sisters, Mort, Guards! Guards, Hogfather, Going Postal, The Truth, and The Wee Free Men.  

Thank you, TNO!

Now, to the miraculous place known only as .... The Library.