'96 @ $3400 - thats a pretty good deal. But the 180K would bother me too. I recently (4 months ago) bought a '93 SC400 with 125K on it.
Your car has probably done most of the miles in more recent years - since they weren't daily-drivers for the people who could originally afford them when it was new. This means you should have the vehicle thoroughly checked out for the high-wear parts (brakes, suspension, door-joints) - these can get to be a pretty expensive job. Not to mention you are on the second timing-belt change, water-pump etc. These cars can be very expensive to maintain when it comes down to it.
That said, they have bullet-proof engines and tend to be very reliable (even CR gives them a top pick - they won best car awards for 3 years in a row back then). I am aware of many people who have purchased these vehicles even when they have clocked over 200K - so the mileage would not concern me, as long as the car checks-out with a good mechanic. Set your expectations with him, about what you are looking to spend in due course of time, and then seek his opinion.
Hope that helps.
Thanks a lot. This is really helpfull.