Non-Nav Audio Confusion

[rmahoneyiv]

I have a 2004 non-nav gx470. I installed an aftermarket radio and a new amp in addition to the factory pioneer one. I wired everything correctly and the amp gets power and should be working fine. However, I noticed that when I unplug the aftermarket amp, there is no noticeable difference in sound quality or volume, but when I unplug the factory pioneer one, no sound comes out at all. This makes no sense to me though because I'm running the speaker wire directly from the head unit's wiring harness to the amp, so there should be no way there would still be audio without the aftermarket amp plugged in. I was wondering if maybe I have to do something and bypass the factory amp, or if I'm just being completely stupid and missing something obvious?