This does have an effect, but I doubt it explains the entire story. I know from personal experience that having to input credit card details into a website, and the delay that input step causes, can often turn me away from finalizing a purchase. Amazon's 1-click proved the value of this sort of process improvement years ago.
This is one of the reasons I've bought so much stuff from Amazon (or used Amazon's payment service). They have my address, they have my card, it's fast as easy.
I don't have to go create a new custom account on someone's store, re-enter that info, possibly make a typo, pull out my CC because I don't have the number memorized, etc.
I'm willing to pay Amazon more than some other stores because, even ignoring shipping, there is much less friction.
