Joe...do not put any confidence in the color indicator. You need to get the battery checked at a dealer or auto store if you can not do it yourself.
It could turn out to be the alternator but chances are it is just the battery, especially if it is more than a few years old. It might also be as simple
as a bad battery cable connection, since you mention that it is fine after being driven. Begin by inspecting and cleaning(if needed) the battery posts and cables.
Thanks for the feedback. I will get it checked out this week. If its the battery, is there any specific place I should get the new battery from? I read the manual and it obviosly recommends getting the battery charged at a Lexus dealer. I guess my question is do I have to get the same battery or will any replacement battery work without ruining any of the cars components?