I had the same problem with my 2004 ES330. After checking quiescent current (~ 35 ma ) and ruling out battery and alternator, it turned out to be a loose negative ground connection at the battery. The clamp was worn and slightly stretched I could move the connection by pushing it. I torqued down the screw clamp and had continuous days of start free operation.