Can't Replace Just The Speakers In My Lx 450?

[RARE]

I went from an Infiniti with a Bose system to my '96 LX 450 with the Lexus Premium Sound system and was greatly disappointed in the lack of quality - not in the truck, I love the 450 just not the audio system. Virtually no bass and even distortion in the highs and mids. I'm not wanting to blow the doors off, so I looked into just replacing the speakers throughout and was told that because of the way Lexus builds their audio you can't do that. It's all or nothing. Is that true? Or has anyone else figured out a way to replace just speakers? What's the deal here?

[SKperformance]

you can replace some parts but it gets very weak after anything is replaced

like a speaker

then the amp cannot push it as it is under powered for any other speaker replacment