I'm merely a lurker on this forum. I recently (couple of weeks ago) purchased a 2004 ES 330, and I'm really happy with the car. After choosing the car on the web, I went looking for problems on the web, and saw the same complaints about the transmission.
I was driving a '95 Honda Accord before and everything is much better (comfort, ride, power). I don't know that I can tell. But I guess you have to decide for yourself. Test drive a couple of cars.
Sometimes I think I know, and I play with manually downshifting as if I can avoid the "problems. But all in all, I don't think I detect it.
I really like my new car, maybe the new car smell is overriding my senses.
-Demian
ps. Drive the car, decide for yourself if you like it, and enjoy.
pps. If "they" fix the transmission with software, I'll take the upgrade, but then I always like upgrades! ;)