You might want to first post in the correct section. Your car model lists ES330, this is the LS400/430 section.
As such the console design is a bit different from what we're used to, but I'm sure the cupholder is secured with screws that are covered by trim somewhere.
I am surprised this showed up here...I thought I did post in the 330 thread.... sorry