It probably has to do with the tension of the serpentine belt. The belt part is temperature sensitive for some reason, and I have the same problem with my '01 ES. Occasionally I have sprayed WD-40 to the belt, and the noise stopped, but of course that's just a temporary fix. I'll have to find some time to order the belt and get it replaced, but as others have mentioned, this won't harm your car in the long run, nor will it affect your starter.