There is a prob in your vapor recovery system. I had same issue on my 02 Sienna. In my case and according to my research the most common cause is the VSV valve on the charcoal canister is bad. I replaced mine as a unit (valve and canister). Dealer can diagnose pretty easily.