like you said, it depends. do you live in a big or small city? do you drive around a lot or just commute, local shopping, and other familiar places? does the driver of the car have map skills? can you afford it?
I had a 2001 RX without Nav. My wife is the primary driver and she is map challenged. We live in a Chicago suburb and she drives to many different places, she used to call me from her cell phone for directions. My dad bought an 02 LS430 with nav and i thought it was a neat toy but not worth the cost. I then bought an RX330 with Nav for my wife - costly and i considered the nav an expensive luxury item - but like you said, the premium ones come with nav.
this gave me a chance to use it and now it has become a 'crutch' like everything else - tv remote, cell phone, email, internet - what would we do without these things? it is so easy to use (i have a bmw with nav that is much more cumbersome) that it has become more than a toy for us. addresses are easy to input. it remembers previous destinations. the travel time estimate feature helps plan (my kids don't ask "are we there yet" because the time remaining is for them to see). i have used the hospital finder in an unknown area. it helps with traffic jam shortcuts. we have been looking for a new house and it's indispensible. restaurants and meeting places are easy to find. the 'home' feature is great, etc, etc.
many will disagree with me ("just use a $2 map!" - i was one who preached this too) any they probably don't need it. so it depends on your situation. good luck.
