I've noticed with my 2008/IS350 that if I keep a tight grip on the sterring wheel in windy conditions it may "feel" like I'm fighting it. Correcting, over-correcting, correcting, etc. I've found a light grip and not worrying too much about the wind seems to work better. I don't know if that makes sense to anyone else. Of course that doesn't mean that will help the OP who may have a legetimate mechanical problem.
JohnCh