My Rx300 has exact same problem, so I got read the forum and took out the 2 air intake hoses and the IACV locates right under them. I took out the two screws and cleaned the inside with the throttle body cleaner. The problem went away, so here is the pictures I took (IACV is the black piece with the two screws, one of them has the yellow dot on it).
Hope this help somebody in the future.