Ok, I actually found it but it is listed in a very deceiving way. Go to Navigation and in the top right is a button to "Delete Destination". I saw it prior but thought that meant to delete it permanently from my Address Book. It keeps the address stored, just stops the Nav. In my opinion it should be reworded to "Cancel Navigation".