I have a 1997 ES300 as well, same problem but the power mirrors also don't work. I've seen several people post this problem, and no one has said if they have a solution. I checked the fuse for the clock, and it was fine.
Please let me know if anyone out there was able to fix this problem?
Thanks,
Matt