OK, I don't have an RX but I'll bet the intake to the cabin filter is the same place as in my 00 LS that has it's cabin filter in about the same location as yours.
Look under the hood just forward of your windshield. This diagram shows the intake on your RX in the same place as it is on my LS:
http://www.toyodiy.com/parts/p_U_2005_LEXU....html?hl=55879BLook in the area of the "SEAL, COWL VENT". Do you see any air grills or even a more major part missing?
I've been surprised at how much debries gets through the little holes in the vent grates and onto the cabin filter of my LS. But acorns? Either its a bad design or something in the cowl vent area is loose or missing on your RX.
The following diagram shows your cabin filter and where it intakes air from the cowl vent:
http://www.toyodiy.com/parts/p_U_2005_LEXU...WAGKA_8714.htmlWe're definitely starting to rely on the toyodiy website.