FAHClient V7.1.38 released (4th Open-Beta)

[Grandpa_01]

Well it is not fixed for me either still 1 hour for Windows to send results of a 6901 and 15 min for Linux to send a larger 6903 /6904 file from the same machine running on a VB. My guess would be a Windows problem.

[Napoleon]

Noticed some weird lines in a recent log, are these something to be concerned about?

Code: Select all

10:37:03:Started thread 14 on PID 5168
10:37:03:Server connection id=2 on 0.0.0.0:36330 from 127.0.0.1
10:37:03:SocketDevice::read(): Receive error: 10053: An established connection was aborted by the software in your host machine.
10:37:03:Server connection id=1 ended

Code: Select all

16:54:34:Started thread 18 on PID 5168
16:54:34:Server connection id=4 on 0.0.0.0:36330 from 127.0.0.1
16:54:35:SocketDevice::write(): Send error: 10053: An established connection was aborted by the software in your host machine.
16:54:36:SocketDevice::write(): Send error: 10053: An established connection was aborted by the software in your host machine.
16:54:36:Server connection id=3 ended

[bruce]

Development has been working on some issues related to communications. The most recent beta is doing better checking of communications links and although this is the first time I've seen the messages, I'm not surprised by their content. If you can associate with them with some events that might influence remote communications, your observations will be appreciated.

[Napoleon]

Well, there was one more exceptional thing, I had opened wudata_01.log file in Notepad from a work directory and forgot to close it before the unit was finished. No editing, but there were some failed attempts in cleaning up that slot. I have no idea if this could somehow have triggered the communications problems, but they appeared after these. It's the only other unusual thing I can think of at the moment. I've restarted FAHControl and FAHClient, will wait and see if I get those communication problems ever again.

Code: Select all

00:07:57:Cleaning up Unit 02
00:07:57:ERROR: Exception: Failed to remove directory './work/02': boost::filesystem::remove: The process cannot access the file because it is being used by another process: ".\work\02"
00:09:34:Cleaning up Unit 02
00:09:34:ERROR: Exception: Failed to remove directory './work/02': boost::filesystem::remove: The process cannot access the file because it is being used by another process: ".\work\02"
00:12:11:Cleaning up Unit 02
00:12:11:ERROR: Exception: Failed to remove directory './work/02': boost::filesystem::remove: The process cannot access the file because it is being used by another process: ".\work\02"

[Jesse_V]

Mr. Coffland,
One thing you might consider as a feature/enhancement request: having the slots in Novice mode be able to be right clicked just as they can in Advanced/Expert mode. Right now, if I want to pause one slot (like the GPU) I have to go to Advanced and then right click it. I can imagine that Novices might want to do the same thing for gaming, other applications, etc, since the SMP backs off pretty well. I understand that menu option also has things like "Viewer" and that there's also a big Viewer button on the top of the window, but I don't see much harm in the minor redundancy for Novice mode as well.

Also, is there a way to change the "Description" of a FAHSlot?
Thanks.

[7im]

You don't think pausing individual slots is more of an advanced level action?

Why not just default everything to Expert level? The same reason that answer is also a no. Novice mode is made simple for a reason. No extra bells and whistles to confuse new users, and no accidental clicks to do things they don't know how to undo. Besides, I'd put most gamers in to categories other than Novice. I don't think they have any problems finding Expert mode.

Although, the presence of that right click is not obvious if you don't read the docs. Almost like the cursor should change slightly when hovering over the folding slots...


Change description of FAHSlot WHERE? From what, to what?

[^w^ing]

Unfortunately, no. I did make some comparisons earlier, though, viewtopic.php?f=18&t=18609&start=15#p186598. Not the exact same WUs, but same server. I've never had issues with v6 upload speeds, whereas there was consistent "throttling" with older v7 beta client. I'm sorry to hear the latest v7 still has upload speed issues in your case. Quite puzzled, too. I would imagine our routes to Stanford are somewhat similar?

Code: Select all

 5     18 ms    19 ms    18 ms  xe-4-2-0-0.psl-peer2.hel.fi.ip.tdc.net [62.237.169.20]
 6    127 ms   126 ms   126 ms  ae-0.nyk2nqp2.us.ip.tdc.net [83.88.21.55]
 7    219 ms   219 ms   219 ms  calren-cenic.paix.net [198.32.176.33]
 8    223 ms   221 ms   221 ms  dc-oak-core1--paix-px1-ge.cenic.net [137.164.47.19]
 9    220 ms   220 ms   219 ms  dc-svl-agg1--oak-core1-10ge.cenic.net [137.164.47.122]
 10   221 ms   221 ms   221 ms  dc-stanford--svl-agg1-10ge.cenic.net [137.164.50.158]
 11   222 ms   221 ms   220 ms  boundarya-rtr.Stanford.EDU [68.65.168.33]
 12   221 ms   221 ms   221 ms  bbra-rtr-a.Stanford.EDU [171.64.255.129]
 13   221 ms   221 ms   221 ms  yoza-rtr-a.Stanford.EDU [171.64.255.144]
 14   221 ms   221 ms   221 ms  vspg2v.Stanford.EDU [171.64.65.64]

well, routes change, especially such long ones. and it shouldn't really matter either way.
what I noticed using network traffic monitor, is that, same as with the old builds I tried, the client's upstream traffic is clearly cut shaped, suggesting that the cause of the slow transfer rate is local to the client rather than being bottlenecked along the route, or by the work server. windows related problem, as Grandpa_01 suggested, would make sense but since you seem to use the v7 client on windows as well, this issue is still as obscure as ever to me.

[Jesse_V]

You don't think pausing individual slots is more of an advanced level action?

Why not just default everything to Expert level? The same reason that answer is also a no. Novice mode is made simple for a reason. No extra bells and whistles to confuse new users, and no accidental clicks to do things they don't know how to undo. Besides, I'd put most gamers in to categories other than Novice. I don't think they have any problems finding Expert mode.

Although, the presence of that right click is not obvious if you don't read the docs. Almost like the cursor should change slightly when hovering over the folding slots...


Change description of FAHSlot WHERE? From what, to what?

Well I have in my mind the spectrum of "novice" to "expert" features in v7, pretty much ranging from pressing the Viewer button on one end to monitoring another computer on the other. Pausing all slots is somewhere in the middle, and it doesn't seem all that complex so I felt like it would have no problem in Novice mode. It wouldn't change anything at first glance about the Novice display: it would still look simple and easy. If they right clicked on one of the slots, then they would see those simple further options. It's the same options as what's on the top of the screen (which are labeled with mouse-over text) so I don't see what's all that complicated about it anyway. In fact, if I try to put myself in a Novice's shoes, I would understand what the right click options do far more than whatever those numbers under "PRCG" mean. I mean I know what PRCG stands for, but its not entirely intuitive. If we don't want to "confuse new users" and make things "simple", how about changing it to "Work Unit Identification"? The right click menu thing makes sense, as when you right click it highlights that particular slot (indicating that one only) and then you see the menu. Obviously the menu applies to that slot, and like I said all the options are already there. I just don't see what's all that complicated about it. If they pause a slot and can't figure out how to unpause it, there's that big "Fold" button that should make everything run.

I like that idea about the cursor changing or something, or some other option to indicate further options for each slot.

One other thing is that when you pause a slot in Novice mode, it changes to "Ready". Ready for what? Why not change it to "Paused". Isn't that more obvious? And then when you unpause it it takes a little bit to get started, so why not have it then read "Starting..."?

Under Advanced/Expert mode, each of the FAHSlots have descriptions. For example mine say "smp:4" and "gpu:0:gt216 [GeForce GT 240M]". I was just asking if there was a way to change this so I could make it look a little more visually appealing, to like "SMP (4)" and "GPU". If not it's not a big deal, I was just wondering if there was such a feature. I mean I know the name of my graphics card, I don't need it to be displayed all the time like that. Just bare and important information about each slot so I can identify which one is which, that's all. Also the semicolons bother me a little bit.

[codysluder]

Maybe we need four settings. In the cases you've described, I think "Paused" is a lot better word than "Ready" and that's true whether you're a novice or not.

For example: Start the client with a slow internet connection. For a short time, the client is busy downloading a new wu and perhaps a new core. Paused might give the wrong impression whereas Ready would be more accurate. Similarly if the servers are out of work, the slot would more accurately be described as "ready" than "paused" because there's nothing that the user can do to get it to start processing.

On the other hand, this would make the client code more complex by making a distinction between Paused and Ready and I'm not sure that's necessary.

[jcoffland]

Noticed some weird lines in a recent log, are these something to be concerned about?

Code: Select all

10:37:03:Started thread 14 on PID 5168
10:37:03:Server connection id=2 on 0.0.0.0:36330 from 127.0.0.1
10:37:03:SocketDevice::read(): Receive error: 10053: An established connection was aborted by the software in your host machine.
10:37:03:Server connection id=1 ended

Code: Select all

16:54:34:Started thread 18 on PID 5168
16:54:34:Server connection id=4 on 0.0.0.0:36330 from 127.0.0.1
16:54:35:SocketDevice::write(): Send error: 10053: An established connection was aborted by the software in your host machine.
16:54:36:SocketDevice::write(): Send error: 10053: An established connection was aborted by the software in your host machine.
16:54:36:Server connection id=3 ended

Those lines indicate that a connection to either FAHControl or FAHViewer was broken. There are many reasons this can happen and it's generally nothing to worry about. However, the last beta had connection dropping problems so if you are using either an FAHClient or FAHControl from a previous beta that could be the reason for the messages.

Well, there was one more exceptional thing, I had opened wudata_01.log file in Notepad from a work directory and forgot to close it before the unit was finished. No editing, but there were some failed attempts in cleaning up that slot. I have no idea if this could somehow have triggered the communications problems, but they appeared after these. It's the only other unusual thing I can think of at the moment. I've restarted FAHControl and FAHClient, will wait and see if I get those communication problems ever again.

Code: Select all

00:07:57:Cleaning up Unit 02
00:07:57:ERROR: Exception: Failed to remove directory './work/02': boost::filesystem::remove: The process cannot access the file because it is being used by another process: ".\work\02"
00:09:34:Cleaning up Unit 02
00:09:34:ERROR: Exception: Failed to remove directory './work/02': boost::filesystem::remove: The process cannot access the file because it is being used by another process: ".\work\02"
00:12:11:Cleaning up Unit 02
00:12:11:ERROR: Exception: Failed to remove directory './work/02': boost::filesystem::remove: The process cannot access the file because it is being used by another process: ".\work\02"

This is something completely different unrelated. In Windows, if you have a file open it cannot be deleted. The client needs to clear the queue slot when it's done and having an open log will prohibit this. This will cause errors in the log file but the client should recover gracefully once you release the file.

[jcoffland]

what I noticed using network traffic monitor, is that, same as with the old builds I tried, the client's upstream traffic is clearly cut shaped, suggesting that the cause of the slow transfer rate is local to the client rather than being bottlenecked along the route, or by the work server. windows related problem, as Grandpa_01 suggested, would make sense but since you seem to use the v7 client on windows as well, this issue is still as obscure as ever to me.

By "cut shaped" I assume you mean you are seeing transfer peaks followed by gaps. My current theory is that on long connections where the latency is high the client's network stack sends some maximal number of packets and then waits for acknowledgement from the other side before sending any more. On close connections the acknowledgement for a packet comes back before the maximum number of packets has been sent. On distant connections the round-trip delay is long enough to chop up the transfer. If this is true it could be solved by allowing more unacknowledged packets on the line before holding up the transfer. This all sound an awful lot like the Nagle algorithm which is described here under TCP_NODELAY. However, I know for a fact that v6 does not use the TCP_NODELAY option and enabling this option could have a negative impact on throughput for everyone.

The client is directly accessing the OS's socket API and not doing any throttling of its own. So the only conclusion I can come to is that it has something to do with how the client is using the socket API. The only difference between v6's usage of sockets and v7's is that v7 uses select() to check when data can be read or written and then does a non-blocking read or write whereas v6 just does blocking reads and writes and uses more threads. I've determined that, although v7 does need to use non-blocking reads, because it doesn't always know how much data it will be reading, it could do blocking writes. In theory, it shouldn't matter but it is possible that non-blocking mode adversely affects the Nagle algorithm.

I'm testing this idea now and will include blocking writes in the next release if it works during testing. Then we can see if it there is any performance improvement for those who are experience slow uploads in v7.

In addition, I'd like to request that reporters please make a clear distinction between "My upload seems slower in v7," and "I've sampled several data points on the same WS for both v7 and v6 clients and there is a consistent difference in upload speed." There are too many variables involved to make claims based a perceived difference. Please provide actual measurements and please be clear that you are stating a suspicion or theory if you don't have the data to back it up.

[Grandpa_01]

6904 on Linux v7 / 7.1.33 As you can see the problem is not present in Linux

Code: Select all

20:47:58:Connecting to assign3.stanford.edu:8080
20:47:58:News: Welcome to Folding@Home
20:47:58:Assigned to work server 130.237.232.237
20:47:58:Requesting new work unit for slot 00: RUNNING smp:12 from 130.237.232.237
20:47:58:Connecting to 130.237.232.237:8080
20:48:11:Slot 00: Downloading 54.57MiB
20:48:17:Slot 00: 4.05%
20:48:23:Slot 00: 14.19%
20:48:29:Slot 00: 21.93%
20:48:35:Slot 00: 32.75%
20:48:41:Slot 00: 46.64%
20:48:47:Slot 00: 62.02%
20:48:53:Slot 00: 77.39%
20:48:59:Slot 00: 92.88%
20:49:02:Slot 00: Download complete
20:49:02:Received Unit: id:01 state:DOWNLOAD project:6904 run:2 clone:47 gen:11 core:0xa5 unit:0x0000001152be746d4e1c217b6d82b405
21:27:31:Unit 00:Completed 250000 out of 250000 steps  (100%)
21:27:44:Unit 00:DynamicWrapper: Finished Work Unit: sleep=10000
21:27:54:Unit 00:
21:27:54:Unit 00:Finished Work Unit:
21:27:54:Unit 00:- Reading up to 121622496 from "00/wudata_01.trr": Read 121622496
21:27:54:Unit 00:trr file hash check passed.
21:27:54:Unit 00:- Reading up to 108764940 from "00/wudata_01.xtc": Read 108764940
21:27:55:Unit 00:xtc file hash check passed.
21:27:55:Unit 00:edr file hash check passed.
21:27:55:Unit 00:logfile size: 216217
21:27:55:Unit 00:Leaving Run
21:27:56:Unit 00:- Writing 230776645 bytes of core data to disk...
21:28:25:Unit 00:Done: 230776133 -> 222430246 (compressed to 3.3 percent)
21:28:25:Unit 00:  ... Done.
21:28:52:Unit 00:- Shutting down core
21:28:52:Unit 00:
21:28:52:Unit 00:Folding@home Core Shutdown: FINISHED_UNIT
21:28:55:FahCore, running Unit 00, returned: FINISHED_UNIT (100 = 0x64)
21:28:55:Sending unit results: id:00 state:SEND project:6903 run:8 clone:3 gen:46 core:0xa5 unit:0x0000003752be746d4de9270f0c320006
21:28:55:Unit 00: Uploading 212.13MiB to 130.237.232.237
21:28:55:Connecting to 130.237.232.237:8080
21:28:55:Starting Unit 01
21:28:55:Running core: /var/lib/fahclient/cores/www.stanford.edu/~pande/Linux/AMD64/beta/Core_a5.fah/FahCore_a5 -dir 01 -suffix 01 -lifeline 1058 -version 701 -checkpoint 15 -np 12
21:28:55:Started core on PID 3556
21:28:55:FahCore 0xa5 started
21:28:55:Unit 01:
21:28:55:Unit 01:*------------------------------*
21:28:55:Unit 01:Folding@Home Gromacs SMP Core
21:28:55:Unit 01:Version 2.27 (Thu Feb 10 09:46:40 PST 2011)
21:28:55:Unit 01:
21:28:55:Unit 01:Preparing to commence simulation
21:28:55:Unit 01:- Looking at optimizations...
21:28:55:Unit 01:- Created dyn
21:28:55:Unit 01:- Files status OK
21:28:58:Unit 01:- Expanded 57219130 -> 71843392 (decompressed 50.4 percent)
21:28:58:Unit 01:Called DecompressByteArray: compressed_data_size=57219130 data_size=71843392, decompressed_data_size=71843392 diff=0
21:28:59:Unit 01:- Digital signature verified
21:28:59:Unit 01:
21:28:59:Unit 01:Project: 6904 (Run 2, Clone 47, Gen 11)
21:28:59:Unit 01:
21:28:59:Unit 01:Assembly optimizations on if available.
21:28:59:Unit 01:Entering M.D.
21:29:01:Unit 00: 0.96%
21:29:06:Unit 01:Mapping NT from 12 to 12
21:29:07:Unit 00: 2.11%
21:29:11:Unit 01:Completed 0 out of 250000 steps  (0%)
21:29:13:Unit 00: 3.20%
21:29:19:Unit 00: 4.38%
21:29:25:Unit 00: 5.47%
21:29:31:Unit 00: 6.55%
21:29:37:Unit 00: 7.26%
21:29:43:Unit 00: 7.93%
21:29:49:Unit 00: 8.54%
21:29:55:Unit 00: 9.36%
21:30:01:Unit 00: 10.06%
21:30:07:Unit 00: 10.75%
21:30:13:Unit 00: 11.45%
21:30:19:Unit 00: 12.15%
21:30:25:Unit 00: 12.84%
21:30:31:Unit 00: 13.53%
21:30:37:Unit 00: 14.24%
21:30:43:Unit 00: 14.86%
21:30:50:Unit 00: 15.64%
21:30:56:Unit 00: 16.46%
21:31:02:Unit 00: 17.15%
21:31:08:Unit 00: 17.82%
21:31:14:Unit 00: 18.51%
21:31:20:Unit 00: 19.24%
21:31:26:Unit 00: 19.79%
21:31:32:Unit 00: 20.50%
21:31:38:Unit 00: 21.32%
21:31:44:Unit 00: 22.01%
21:31:50:Unit 00: 22.72%
21:31:56:Unit 00: 23.41%
21:32:02:Unit 00: 24.11%
21:32:08:Unit 00: 24.79%
21:32:14:Unit 00: 25.38%
21:32:20:Unit 00: 26.20%
21:32:27:Unit 00: 26.82%
21:32:34:Unit 00: 27.67%
21:32:40:Unit 00: 28.52%
21:32:47:Unit 00: 29.17%
21:32:54:Unit 00: 29.99%
21:33:00:Unit 00: 30.83%
21:33:06:Unit 00: 31.52%
21:33:12:Unit 00: 32.19%
21:33:18:Unit 00: 32.89%
21:33:24:Unit 00: 33.58%
21:33:30:Unit 00: 34.14%
21:33:36:Unit 00: 34.85%
21:33:42:Unit 00: 35.65%
21:33:48:Unit 00: 36.35%
21:33:54:Unit 00: 37.05%
21:34:00:Unit 00: 37.73%
21:34:06:Unit 00: 38.41%
21:34:12:Unit 00: 39.08%
21:34:18:Unit 00: 39.60%
21:34:24:Unit 00: 40.49%
21:34:30:Unit 00: 41.18%
21:34:36:Unit 00: 41.88%
21:34:42:Unit 00: 42.58%
21:34:48:Unit 00: 43.28%
21:34:54:Unit 00: 43.96%
21:35:00:Unit 00: 44.53%
21:35:06:Unit 00: 45.37%
21:35:12:Unit 00: 45.98%
21:35:18:Unit 00: 46.78%
21:35:24:Unit 00: 47.48%
21:35:30:Unit 00: 48.17%
21:35:36:Unit 00: 48.87%
21:35:42:Unit 00: 49.55%
21:35:48:Unit 00: 50.28%
21:35:55:Unit 00: 50.82%
21:36:01:Unit 00: 51.71%
21:36:07:Unit 00: 52.49%
21:36:13:Unit 00: 53.18%
21:36:19:Unit 00: 53.88%
21:36:25:Unit 00: 54.57%
21:36:31:Unit 00: 55.26%
21:36:37:Unit 00: 55.89%
21:36:43:Unit 00: 56.55%
21:36:49:Unit 00: 57.24%
21:36:55:Unit 00: 57.93%
21:37:01:Unit 00: 58.63%
21:37:07:Unit 00: 59.33%
21:37:13:Unit 00: 60.02%
21:37:19:Unit 00: 60.73%
21:37:25:Unit 00: 61.42%
21:37:33:Unit 00: 62.10%
21:37:39:Unit 00: 63.04%
21:37:45:Unit 00: 63.54%
21:37:51:Unit 00: 64.33%
21:37:57:Unit 00: 65.08%
21:38:03:Unit 00: 65.78%
21:38:09:Unit 00: 66.48%
21:38:15:Unit 00: 67.18%
21:38:21:Unit 00: 67.87%
21:38:27:Unit 00: 68.54%
21:38:33:Unit 00: 69.25%
21:38:39:Unit 00: 69.98%
21:38:45:Unit 00: 70.67%
21:38:51:Unit 00: 71.37%
21:38:57:Unit 00: 72.06%
21:39:03:Unit 00: 72.76%
21:39:09:Unit 00: 73.47%
21:39:15:Unit 00: 74.02%
21:39:21:Unit 00: 74.82%
21:39:27:Unit 00: 75.50%
21:39:33:Unit 00: 76.20%
21:39:39:Unit 00: 76.89%
21:39:45:Unit 00: 77.58%
21:39:51:Unit 00: 78.25%
21:39:57:Unit 00: 78.83%
21:40:03:Unit 00: 79.51%
21:40:09:Unit 00: 80.23%
21:40:15:Unit 00: 80.91%
21:40:21:Unit 00: 81.74%
21:40:27:Unit 00: 82.45%
21:40:34:Unit 00: 83.05%
21:40:40:Unit 00: 83.96%
21:40:46:Unit 00: 84.65%
21:40:52:Unit 00: 85.34%
21:40:58:Unit 00: 86.04%
21:41:04:Unit 00: 86.75%
21:41:10:Unit 00: 87.44%
21:41:16:Unit 00: 88.14%
21:41:22:Unit 00: 88.84%
21:41:28:Unit 00: 89.53%
21:41:34:Unit 00: 90.24%
21:41:40:Unit 00: 90.92%
21:41:46:Unit 00: 91.64%
21:41:52:Unit 00: 92.14%
21:41:59:Unit 00: 93.01%
21:42:05:Unit 00: 93.84%
21:42:11:Unit 00: 94.54%
21:42:17:Unit 00: 95.25%
21:42:23:Unit 00: 95.95%
21:42:29:Unit 00: 96.49%
21:42:35:Unit 00: 97.19%
21:42:41:Unit 00: 98.05%
21:42:48:Unit 00: 98.65%
21:42:54:Unit 00: 99.56%
21:43:24:Unit 00: Upload complete
21:43:24:Server responded WORK_ACK (400)
21:43:24:Final credit estimate, 286409.00 points
21:43:24:Cleaning up Unit 00

[Jesse_V]

Another interesting feature for v7 would be some kind of update alert feature. Some other software I have does this, it has a little popup that says "A new version is available, would you like to download it?" and then you press Yes and and it does. That simplifies things for the user I think, since even the best software gets updates every now and then, and I don't think v7 will be an exception. Even some kind of self-update thing upon launch (like Minecraft does) would be useful. That way after v7 goes public, cores/fixes/whitelist/etc updates can still be easily applied and everyone benefits even if they aren't following Dr. Pande's blog all the time.

I just thought that'd be a good idea.

[Adam A. Wanderer]

Another interesting feature for v7 would be some kind of update alert feature. Some other software I have does this, it has a little popup that says "A new version is available, would you like to download it?" and then you press Yes and and it does. That simplifies things for the user I think, since even the best software gets updates every now and then, and I don't think v7 will be an exception. Even some kind of self-update thing upon launch (like Minecraft does) would be useful. That way after v7 goes public, cores/fixes/whitelist/etc updates can still be easily applied and everyone benefits even if they aren't following Dr. Pande's blog all the time.

I just thought that'd be a good idea.

One thought, as an active user is constantly connecting/disconnecting from F@H servers, it could report the version to the server. If no later version is available, nothing happens. If a new version is available, an e-mail or popup is sent indicating the availability and do you want to upgrade, or not. If "Yes", then instructions are sent to upgrade or the new version over writes the old version.

[Ivoshiee]

It seems that fahclient-7.1.38-CentOS_5.3_64bit_release.x86_64.rpm is not setting the FAH client up as a service under Fedora Core 15.
I had to issue

Code: Select all

chkconfig --add FAHClient

to get it listed (chkconfig --list).