This is my fix - '92 ES300
The driver’s side mirror swings back and forth and will not remain in the detent where it used to firmly rest. Inside the mirror housing, a “U” shaped clip that kept a spring compressed enough to hold the mirror in place worked its way out of place. The plastic cover is too small to allow compressing the spring and reattaching the “U” clip. I finally discovered a very inexpensive fix. I bought a square piece of rubber about 5” x 5” from a crafts & hobbies store. This flexible piece of rubber is about 1/8” thick. With common scissors, I cut a square about 1.5” x 1.5”. Swinging the mirror fully toward the front of the car, I placed the rubber square in the vertical gap between the body of the mirror and the base that attaches to the car door. Then pulling the mirror backward towards its proper position drew the rubber square fully into the gap, holding the mirror firmly in place. No tools, and it holds like it did when it was new. You can still adjust the mirror forward and back, and I can’t even see the reddish rubber square. What a laugher, especially since I have agonized for so long about fixing this problem without spending a small fortune. Just make sure the thickness of the rubber is just slightly thicker than the gap between your mirror housing and the part that attaches to your door.
Probably simple garden hose washer would work too. Whatever works for the driver’s side mirror will certainly work on the passenger’s side mirror too.
Hope this helps