Well, at last I got it done after using some force to the area where the bracket and the pan meet. I still believe there is another way to get the pan off. I know Toyota engineers, they don't trick (if you ask someone who owns Infinity Q45, you'll know those engineers don't want YOU to fix their cars easily).
Now, the pan is in its place with a new gasket and with one screw was over tightened (Oops!). I am crossing my fingers. I hope not seeing any leaks. Wish me luck. By the way, when you place the pan back, make sure that the filling tubes are one inside another before you tighten the bolts.
I did change my fuel filter too. This time, I'm so glad that I did. Gasoline came out looked like mud. I thank you for those people who work hard to make this forum so educated. I don't even know who they are but I learn and enjoy a lot getting all the knowledge about my Lexus.