I did the same. I bought a used 2001 in good condition and felt that "hey it is a Toyota product so enough said". I have had it now 1.5 years with no problems and then came across this forum along with a couple others and started to read about them and immediately felt like crap. I am still sorting through all this new info coming my way.
Here is my thoughts and .02 worth on this. I have checked with a couple transmission shops that do high volume and also with an independant Toyota/Lexus repair shop. All three said that they see nothing unusual on these. One of the largest tranny shops advised that they do maybe one RX300 transmission rebuild every couple months. The common thread from all is to change your fluid regularly with the Toyota type IV fluid. Most times when these take early retirement is when the fluid was never changed. That is what they told me anyway. I am sure some folks have had issues even doing the fluid changes and that can happen with any car. You sometimes get one that does you wrong. I have had Acura's that I put 175k miles on both without ever having a problem with the tranny but I changed the fluid actually once/year because I live in a very hot climate. I have heard others with Acura's who did not change the fluid and they were toast at 100K. Just change the fluids. It is not a guarantee but I think it is a guarantee that you will have early failure if you do not. That goes for the oil as well on your sludge concerns.
Since you just bought this I would recommend either pulling or having someone pull a valve cover and take a look, and do it very soon. If it looks good then change the oil regualrly and I doubt you will have a problem with sludge. If there are signs of trouble then take it back immediately to the Lexus dealer and ask for a new engine. Since you just bought it I would bet that they will rectify this for you. I think they extended the warranty for this anyway till 100K so they should cover this under goodwill however I would act on this now. If it is ok then you can sleep again at night. On the tranny, all you can do is change the fluid and hope for the best. Maybe have someone pull the pan to make sure the filter is clean and see if there are signs of any unusual metal in the pan. If so then take issue with this now too with the dealer before more time passes.
I suspect on the tranny's that when used in a Camry they are fine but when used in a heavy vehicle like the RX it stresses it. If you have a an AWD then that adds to the stress and heat. As mentioned above I would not tow or work the vehicle as a truck, rather treat it more as a car and your odds of having major problems goes down.