This just happened to my '05 LS430. The tl:dr version of my story is a specialist company rebuilt it in town for ~half the price of a new part (~$600).
When my problem started, I checked every ground and fuse with a multimeter and couldn't figure it out. I brought it to my trusted local shop, who couldn't figure it out. They outsourced it to a local vehicle electronics specialist that wouldn't even touch it. Apparently, the bulbs in this part get so hot and just weaken the plastic over the years and makes them very brittle to remove and work on. I finally took it to the dealer for a replacement, but the part has been discontinued and they couldn't find one in stock anywhere. They subbed it out to the rebuilder who did the job in about 2 weeks. While it was there, I asked the dealer which specific part it was (83801-50200, 83801-50201, or 83801-50221) so I could start looking for used ones, but couldn't get a straight answer.
I hope this helps future owners who have this problem and find this thread!