Sure there's a litle delay, it's not a Ferrari SMG. However there's a delay when you shift on a manual gearbox as well, except the driver's causing it. The IS box is actually pretty darn fast.
Personally after thinking the paddles were a good idea, since I got my 350 I find myself using the stick more than the paddles. Or just leaving it on auto with ECT in PWR mode. The way I've described it to friends, in standard mode the car feels like a (fast) Camry, in ECT-PWR it turns into a psycho!