#682 folding-2.pdc.kth.se: Noticeable V7 slowdown

[k1wi]

Is it possible to -oneunit the v7client and see whether a v6 client has the regular result? Or have you run 690*'s in the past on the v6 fine?

[bruce]

K1wi:
FYI
In v7, the -oneunit option is called "Finish" and it can be done to all slots or selectively to any one if you're running more than one (such as SMP + GPU), so it's certainly possible.

[k1wi]

Oops sorry, I meant to direct that at Grandpa, to see if he was willing to -oneunit it and switch over to the v6 client.

[Grandpa_01]

Windows v6 had 10min. upload times on the 690x WU's. I will switch over to v6 on the next WU and check it out though it has been a long time since I ran v6 on Windows.

[k1wi]

So far I've only had a regular SMP work unit report back using the v7 client. Upload speed to Stanford from here in New Zealand was comparable to v6. Ended up receiving the first 2689 I've had for a long time next however, so will have to wait further until I get a Swedish work unit.

k1wi

[Grandpa_01]

I have a 6900 running on the same rig using Windows v6 it will complete in about 9 hours so we will see if there is any difference between v6 and v7 then.

[Grandpa_01]

It does not happen with Windows v6 it took 6 1/2 minutes to send a 6900 to the Sweedish server using the same machine.

Code: Select all

[13:02:15] Completed 245000 out of 250000 steps  (98%)
[13:22:50] Completed 247500 out of 250000 steps  (99%)
[13:43:23] Completed 250000 out of 250000 steps  (100%)
[13:43:35] DynamicWrapper: Finished Work Unit: sleep=10000
[13:43:45] 
[13:43:45] Finished Work Unit:
[13:43:45] - Reading up to 52713120 from "work/wudata_01.trr": Read 52713120
[13:43:45] trr file hash check passed.
[13:43:45] - Reading up to 42794504 from "work/wudata_01.xtc": Read 42794504
[13:43:45] xtc file hash check passed.
[13:43:45] edr file hash check passed.
[13:43:45] logfile size: 199658
[13:43:45] Leaving Run
[13:43:50] - Writing 95875222 bytes of core data to disk...
[13:43:51]   ... Done.
[13:44:06] - Shutting down core
[13:44:06] 
[13:44:06] Folding@home Core Shutdown: FINISHED_UNIT
[13:44:09] CoreStatus = 64 (100)
[13:44:09] Sending work to server
[13:44:09] Project: 6900 (Run 32, Clone 18, Gen 48)


[13:44:09] + Attempting to send results [May 19 13:44:09 UTC]
[13:50:41] + Results successfully sent
[13:50:41] Thank you for your contribution to Folding@Home.
[13:50:41] + Number of Units Completed: 132

[13:50:47] - Preparing to get new work unit...
[13:50:47] Cleaning up work directory
[13:50:47] + Attempting to get work packet
[13:50:47] Passkey found
[13:50:47] - Connecting to assignment server
[13:50:48] - Successful: assigned to (130.237.232.141).
[13:50:48] + News From Folding@Home: Welcome to Folding@Home
[13:50:48] Loaded queue successfully.
[13:51:35] + Closed connections
[13:51:35] 
[13:51:35] + Processing work unit
[13:51:35] Core required: FahCore_a5.exe
[13:51:35] Core found.
[13:51:35] Working on queue slot 02 [May 19 13:51:35 UTC]
[13:51:35] + Working ...
[13:51:35] 
[13:51:35] *------------------------------*

[k1wi]

Interesting.

I guess what we need now (in lieu of me receiving and returning a v7 bigadv work unit) is some input from Kasson or jcoffland as to whether this is a result of a difference in the way v7 manages uploads, or a server issue just affecting v7.


Apart from me returning a v7 6900/6901 work unit, I think we've about isolated as much as we can regarding the issue.

[bruce]

I think the only way we're going to be able to gather accurate data is to run both V6 and V7 simultaneously. I propose that somebody configure a Quad with both a V6 and a V7 client. Have them both set for -smp 2. Get lucky enough to be assigned two WUs from the same project. With a little creative settings of affinity, it should be possible to slow down whichever one is ahead of the other one so the can both finish at the same time and both upload concurrently.

That should provide a side-by-side comparison of uploading a result from V6 and a result from V7 with precisely the same internet connection. We'll be able to see if there is any systematic difference.

That's not an easy thing to do, and I'm certainly open to other suggestions, but it's the only think I can think of that really addresses the question about whether V6 and V7 are different. Everything else compares uploads under different conditions.

[k1wi]

Notwithstanding my results (other side of the pacific to California and opposite side of the world to Sweden), I think that is going to take a 16 core machine, as only bigadv are being sent to Sweden... Plus I know the v6 can reduce all non-FAH internet traffic to a standstill while uploading without QOS, so you'd also have a significant potential interaction occurring between clients (plus a fast enough upload to accommodate both clients simultaneously).

I understand that "different conditions" exist, but if there is a consistent pattern, time after time as is appearing to occur, it should probably be investigated by Stanford. All we can do as users is basically continue what we already have done. I suspect that Stanford are probably the only people capable of the resources to undertake a test as you suggest... Hopefully our observations and tests have aided their trouble shooting...

[Napoleon]

Could it be that the developers are experimenting with some QoS features already?

[bruce]

k1wi: I realized we were talking about bigadv after I posted that so, yes, it's going to take a big machine, though perhaps not 16 cores. What it will take is someone who is willing to give up a bonus -- or most of it. When a WU finishes, the client can be blocked from uploading by either aborting the upload or by simply disconnecting from the internet. After a WU has finished on each client, but not uploaded, it's pretty simple to reconnect to the internet and start both clients at the same time. The V6 client can be started with -send all, which will prevent the download of a new WU. The V7 client can be started with pause-on-start=true which should still attempt to send results that are in queue. Setting up WUs that are ready to upload is moderately complicated, but it should be easy to start both clients within a second of each other. and if anything doesn't go as planned, yank the internet cable and start over.

[bruce]

Could it be that the developers are experimenting with some QoS features already?

Not that I'm aware of. That ticket is listed as Milestone: Public Release which is still quite some time in the future. Things listed as Milestone: 7.1.25 are near enough that they are working on them in the lab (or maybe even Milestone Open Beta Phase 2) but certainly not in a client that has been distributed yet.

[GreyWhiskers]

Don't we need to make sure we understand which side of the transfer has the bottleneck? If ONE bigadv transfer hits the sender's upload limits, then TWO simultaneous transfers would just force sharing the outgoing bandwidth.

I have a fairly fast Comcast cable internet connection. The upload speed is slower than the download speed, and there is an allowance for a higher speed "initial burst", which then evens out to whatever caps that Comcast has put on my modem.

HFM is showing that the average uplink for the GPU transfers is about 462 KiBytes/sec, and the average transfer time is about 6 seconds for about 2.5 Mbytes.

HFM shows the average uplink for bigadv transfers is about 272 KiBytes/sec, and the average transfer time is about 366 seconds for about 100 Mbytes

The difference in upload speed could be because my internet service provider throttles the uplink for the really large transfers - say after 10 seconds of max effort, or that Comcast would let me transfer at 462 KiBytes/sec, but the receiver is the bottleneck. I don't know.

Bottom line is that before we try to have someone simultaneously upload two 100 Mbyte data sets, they understand whether their local internet service is the throttle or the Stanford server.

[bruce]

Bottom line is that before we try to have someone simultaneously upload two 100 Mbyte data sets, they understand whether their local internet service is the throttle or the Stanford server.

I agree.

My point is that if we believe this is a V7 only problem, then we'll need to demonstrate that to be true rather than something else causing the problems. Demonstrating it can become rather complicated. There are many possibilities for places where throttling might occur and we really don't have a handle on which is causing the reported slow-downs.