One WU, 2 GPU

[Ricky]

As I understand it, using as many CPU cores as possible for a WU is best for the science; rather than breaking the CPU into many slots. Has anyone thought about using more than one GPU for a WU? Perhaps sync problems would prevent this.

[Joe_H]

Yes, it has been thought about. Syncing was the problem, as well as data transfer speeds between GPU's. Perhaps someday these issues will be resolved, but it is not something I would expect anytime in the near future.

[JimboPalmer]

In the CPU case, memory used by other CPU Cores is as easy to access as memory used by this CPU. For most GPUs, they are very distant, the memory on your card is right there but to transfer to the CPUs memory requires pushing data over the slow PCI-E bus to main memory, then pushing it over the PCI-E bus again to the memory of the other GPU. This can easily be 1000 times slower than in the multiple CPU example.
A GTX Tian Z can access its own memory at 672 Gigabytes a second http://www.geforce.com/hardware/desktop ... ifications

PCI-E 1.0 is theoretically 250 Megabytes a second per lane, (and practical can approach 80% of that) PCI-E 2.0 doubles that to 500 Megabyte per second per lane, (still 20% overhead) and PCI-E 3.0 can do 980 Megabytes per second per lane. (and overhead drops to 1.5%) Even with the latest, greatest, versions you are theoretically limited to 15 GB/sec compared to 650GB/sec on the card itself. http://www.techpowerup.com/reviews/Inte ... ing/1.html

Latency is even worse than Bandwidth, we have to pass through the PCI_E bus twice, and co-ordinate the CPU with both GPUs. (many GPU WUs spend several minutes of CPU time passing data through the PCI-E bus before the GPU becomes busy)