> Their imaginations were definitely encourage by exponentialism, but in fact all they knew was that when the went from smallish to largish networks following the architectural diagram above, the performance got much better. So the inherent reasoning was that if more made things better then more more would make things more better. Alas for them it appears that this is probably not the case.

I recommend reading Richard Hamming's "The Art of Science and Engineering." Early in the book he presents a simple model of knowledge growth that always leads to an s-curve. The trouble is that on the left, an s-curve looks exponential. We still don't know where we are on the curve with any of the technologies. It is very possible we've already passed the exponential growth phase with some of these technologies. If so, we will need new technologies to move forward to the next s curve.