I'm currently running a PS game (3.5, it's not hard to convert), and the most important thing, imo, is to get the feel of the game going. PS is not just another D&D setting - it's got a LOT of flavour to it.

Grab the campaign setting for sure, and "In the Cage" is a must. The third book I'd recommend is "The Planewalker's Handbook", which I'm shocked nobody has mentioned so far.

Beyond that, I'd recommend, in order:
The Factol's Manifesto (political info, and TONS of backstory)
Monstrous Compendiums (I,II,III)
Planes of Conflict
Planes of Chaos
Planes of Law
Hellbound
A Guide to the Astral
Faces of Evil
Uncaged: Faces of Sigil
The Inner Planes
A Guide to the Ethereal
On Hallowed Ground
.. adventures (as others said, skip Faction War)

My copy of 'Player's Primer to the Outlands' doesn't have the CD, so I'm not sure whether I can recommend that. The book itself isn't anything special.

Imo, the adventures are great for getting ideas about flavour, but not particularily good adventures in and of themselves. Most of them depend heavily on players doing certain things at certain times, and we all know how often that happens

For folks looking for 3rd edition stuff, check Welcome to Planewalker | Planewalker. They use the post-Faction War setting (boo!), but there's a lot of info and some great ideas there.