Y'know, I was thinking about this as I was driving around during lunch and I realized that, faster or not, performing a proper downshift is just more satisfying than punching a button or pulling a paddle. To know that you've properly blipped the throttle and are now in the right gear, ready to utilize the powerband, well it's kind of like the car's patting you on the back, y'know? Doing it under brakes and revving down through three gears as you come down off a freeway is like getting an A+ on your extra credit work.
I do really want a clutchless manual someday like DSG, but I'll miss the rewards you get from driving a stick.