Oh, I could have sworn I saw LX 470 on your profile. :)
Anyway, there is a speed sensor involved in the power steering that needs to be checked out. Not sure exactly how that works. Wandering could be due to alignment or worn parts in the steering assembly. It sounds like you need to get that entire front end checked out. If we knew the year, maybe could find if there are any TSB's on that particular model.