There should be only two bolts holding the caliper to the steering knuckle. They are really well hidden, 17mm if memory serves me, and should not be too hard to take off. Just steer your wheels to one side all the way, or the other, and you should be able to get a breaker bar in there, so it sticks out of the wheel well. Other bolts, whatever it is you are seeing, do not hold the brake caliper in place.
If you still cannot take off the caliper with those two bolts removed, you just need to jiggle it a little, the caliper is heavy, and is sitting thight in there (with brake pads possibly hugging the disc). Here I found a picture of the only two bolts that need to be removed:
http://www.clublexus.com/forums/performance-and-maintenance/171570-uh-turbo-time-na-t-feed-me-24.html
Just look at the third image after the black SC400 on the top of the page. First its the black SC, than its engine, than something else than the picture of the same caliper you are working on with those two 17mm bolts that need to be removed indicated. It is fairly simple, they are just hard to see as they are on the back and well hidden, kinda deep inside, you have to stick your head inside the wheel well to be able to see them - I mostly worked by touch only as it was hard to see them, but they are there.
BTW, the screw that you are trying to take off, is a "torx" type of screw, and does not need to be taken off. Usually when you see torx, it means - "We don't want you to touch this!" :)