The plastic part, if broken, can be hard to insert. I have used insolated wire to pull through the hole with the new part on it and a knot on the end. You can not put your hands into the space to work parts together. Since I have no Lexus dealer on isalnd where I live, I fount a GM part that fits. I can give you specs. if you need them. I had to do both of my front door, the driver's side twice, because it slipped out when people grab the handle and yank it too far up. I have also put a tie wrap arround the rod and new plastic insert. It requires a surgical like procedure however. To remove the door pannel is fairly easy after you figure out all the screws and pop of the plastic ring arround the handle.
Good luck.