It is totally worth changing all the speakers. I had the pioneer premiere package and changed them all out to 5 1/4" with components in both front and rears and WOW, what a difference. I also changed the deck and am running off a 4-channel amp. The install job is huge, it took me and 2 buddies 10 hours to complete the project.
I can't wait to replace the stock sub and get a 5 channel amp... maybe next spring, my baby goes in the garage for the winter.