lizzybluts,
My wife and I always purchase 3 to 4 year old Lexus ES's and Subaru Outback's, the pamper mobile for her, and the go through any snowfall one for me.
At this point we have had 4 ES's and every one was better than the one before. The mileage you are looking at would not be a concern for me, as long as the car screams "pride of ownership". Being a backyard mechanic who's been in the backyard for years, I do all of the maintenance and inspections.
In your case, if you have doubts, take the car to an independent mechanic you trust, and have it inspected. It may prove to be the best $100 you'll ever spend.
Any reputable dealership or private individual who won't allow you to get the car inspected privately, has something to hide. Walk away from the car.
If, after the inspection, the verdict is great, buy it if its priced right.
The only things I have ever had to fix over and above regular maintenance on any Lexus we've owned since 1997, was a torque converter on one, and a suspension sensor for the motorized headlights, and an accelerator position sensor on another, period. That's 4 different ES's, each one a different generation, and many thousands of kms collectively. If they've had regular maintenance and haven't been cowboyed, they are pretty much bullet proof.
However, if a major part goes wrong, like most upscale makes, the cost can be substantial. That makes the mechanic's inspection all the more necessary if you don't have the expertise to do it yourself. Good Luck with your shopping!