Still kind of strange though, you shouldn't really have to do that. Maybe they weren't on properly when you got the car. I know a lot of people remove them and only put them on when the need to attach ski racks or a cargo box. I'd try to put them back on your self, and ensure that they are on proper and tight, and I bet the noise will no longer be there.
my .02
I tried moving them back and forth to different positions with no help. The removal was the only thing that worked.