Original key fob with new batteries works sometimes

[Sujit DMello]

Hello,

Noticed the key fob for my 2007 GX 470 wasn't working so replaced the battery with a new one. Unfortunately, even that didn't work even standing right next to the door. So opened the car with the key and ran an errand. After that the key fob works unlocking and locking the car as usual. But after some time (maybe an hour or so) it stops working. If I use the car again, it repeats this sequence.

Has anyone seen this behavior? I can't seem to make sense of it. It is as if the locking system in the car has a battery of its own.