Buddy I've detailed hundreds of cars, what you're saying is not exactly accurate. The dash material on the RX300 is dyed, its not solidly the interior color all the way through. You're right by saying that the eraser dulled the surface, it did that by abrading and removing the top layer of the surface and exposing a fresh layer underneath. The vinyl is impregnated with a UV treatment but only on the surface. When you remove that surface, and potentially that UV layer you just have no idea what effect thats going to have on the longevity of the material, on color fading, etc. IMHO its not worth the risk until you try something else first.
The credo of any responsible automotive detailer is to use the least aggressive product and technique possible to achieve the desired result. Going straight to the absolute most abrasive product you could use isn't the proper approach.
Your statement that there aren't many chemicals you would use on the dash makes no sense when you are using one of the harshest chemicals around...in the Magic Eraser and its not even formulated for that purpose. There are MANY safe chemicals expressly designed to clean automotive vinyl...and do it safely.
When you yourself used the Magic Eraser on your '08 you state it ruined the dash and you had to re-dye it. It may have done the same thing to your '03, you just couldn't tell because the '08 has a black dash and the '03 had a lighter ivory dash.
Armor All is no magic detergent resisting substance, its an ordinary water based dressing, its not even silicone based. Its very easily removed.
The radio bezel is flat black, but its very easily marred. Again, I would never do that to my car or a client's car.
Take whichever advice you want, I've been detailing cars for over 15 years, I've detailed professionally. The Magic Eraser is simply a "last resort" tool. Try Vinylex, 1Z's interior cleaner, or a simple solution of 5:1 water:woolite.