Found this on another Lexus site:
Ok here it goes.
First step: Remove the door Panel
This is possibly the hardest part, and may require a helper to hold the panel so you dont damage the speaker or pull any cables out of their connectors.
You need to first unscrew the panel from the metal part via a couple of screws. Most of them are concealed under plastic panels to preserve aesthetics. 1 is under a panel behind the door pull. Pry it off carefully with a flathead screwdriver. The hinge point is towards the front of the car so pry near the lock. 2 more are located behind a panel at the edge of the vinyl lining. Pop that sucker off carefully with that flathead. 2 more are located underneath the arm wrest, and 2 more are. There are a few more (im going to look tomorrow and edit this because i cant remember exactly where they are located) remove these as well. The door panel will still be stuck however because there are plastic fasteners that hold it on as well so dont try and force the panel off you might break something
This next part of removal is kinda tricky. You need to pop the fasteners out of their holes, and the best method i have found is to put a wood shim between the metal and the panel, and pull gently at each fastner with a crow bar. However you do this you should be careful because Im sure you dont want to scratch your metal.
After you have the panel freed, dont lift it away just yet because there are connectors that work your electronics and door opening/closing. This is where it is nice to have a helper to hold the panel as you undo all the connectors. The connectors are as follows:
Seat Memory unit
Door handle cable
Window switch unit
Mirror adjustment unit
Convience light unit
Each of the connectors for the electronics can be removed with a very small flat head screwdriver. Each connector has a clip that keeps it from coming loose with vibration. Push on the clip to disengage it from the receptical. The most tricky connector was the window switch unit. This one caused me a bit of a headache, but stick with it, youll get that sucker out!
There are 2 cables for the Door handle and the Lock mechanism. The door handle cable stow is a snap fit, and just needs to be popped out. The end of the cable has a small T shaped bar that fits into an appropriate receptical that is mounted on the door handle. Your effort in pulling the door handle pulls that cable, and opens the door. On the lock mechanism, there is a simlar T shaped cable, but the stow is adjustable. Undo the 2 clips on the clearish plastic barrel with your handy flathead and pull that thing out. Now you are done removing the door panel and done with the hardest part!!!!!!!
Step 2: Now we see the problem
First thing you will notice is your speakers (now you know how to undo the door panel if you want to replace your stock speakers later on! ) Second thing is a clear plastic cover to prevent any water from ruining your panel. Get a good grab on that plastic and peel it off. It is held by a gummy black sealing adhesive, and you can reuse it, so be careful not to bunch up the plastic, or scrape off the adhesive. Now you can gaze into that dark chasm that is your inner door. At this point if you need to drive your car you still can, you can lock the door and open it from the inside if you pull on the door handle cable. (I did, its pretty ghetto )
Ok so now you can access the outer door handle mechanism and the problem. I need to explain how the handle mechanism works at this point. Basically there is a lever that is connected to the handle. As you pull it up, this pulls a cable which opens your door. The end of the cable is a hard bent metal piece that is cliped into the handle with a yellow plastic piece