RX300 Shifter Knob Replacement

[Hobiemarty]

I have a 2000 RX300 that has a leather shifter knob and I have found a nice wood and leather shift knob that I would like to swap out. Does anyone know how to disconnect the OD at the button in the side of the shifter knob? I see that there is a piece where the buttons for the OD and shifter release that looks like it can be removed and I have tried to pry that part off with a small flat blade screwdriver and also panel removal tools but it just will not come off and I don't want to break anything.  Any suggestions will be greatly appreciated. 

[DannyLXD]

Howdy! Have you successfully replaced the knob?