If your paint is in exceptional conditions (little to no swirls, zero oxidation, no hairline scratches etc) then by all means do save time and just use the clay. Although, if your car is in need of claying then it is highly likely that your paint has other imperfections that the clay will not remove.
As far as I know, no clay bar that I've used ever removed swirls or fine scratches from the car. I always had to follow the clay procedure with a heavy duty paint cleaner. Just in my experience, though.
The only way to know for sure is, after you clay the car inspect it thoroughly under good lighting. If there are any swirls or scratches, hit it with a heavy duty abrasive cleaner polish.