Not enough a7 WUs

Moderators: Site Moderators, PandeGroup

Re: Not enough a7 WUs

Postby DrBB1 » Wed Feb 28, 2018 8:43 am

Joe_H— Thanks for the update and clarification.
========
DrBB1
DrBB1
 
Posts: 159
Joined: Wed Mar 26, 2008 12:30 am
Location: SE PA

Re: Not enough a7 WUs

Postby bruce » Wed Feb 28, 2018 5:13 pm

The FAH Consortium does not establish preferences (either for or against) individual servers or individual projects but attempts to make the best use of the resources that people donate. Inasmuch as a CPU slot can run either a4 or a7 projects, the only preferences that might be established are designed to share the resources in a semi-equitable manner.

You may (or may not) be right about there currently being a preference for projects from Temple ... to make up for the recent problems that kept them from getting their fair share of donated resources. (or, as Joe has suggested, it may be because there's more work there to be assigned-- in fact, at this moment, 87% of the available WUs are there.) In any case, as new projects are brought on-line, things will change.

Neither of those possibilities are a preference for or against Core_a7, as suggested by your title.
bruce
 
Posts: 21543
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: Not enough a7 WUs

Postby DrBB1 » Thu Mar 01, 2018 1:09 am

bruce-- Don't know if you were responding to anything I wrote (I suspect not), but my interpretation wasn't that there was a "preference" being given. I'm perfectly OK with the fact that more a4s are available than a7s. My interpretation was that to many of us there was no obvious reason for the "sudden" drop in a7s we were receiving. And of all the possible explanations one might be an undetected bug in the distribution system (unlikely, but still...). Another, at least for me, was that because I am located in SE PA--near Temple--that a new assignment algorithm that took ISP location into account might have been implemented (also unlikely, but more reasonable).

At this point in the discussion, I see no reason to be concerned about the recent plethora of a4s being the result of anything more than "the normal course of business." I trust others do as well. [BTW, two of my four new WUs since yesterday were a7s, so the new ones are beginning to kick in.] :-)
DrBB1
 
Posts: 159
Joined: Wed Mar 26, 2008 12:30 am
Location: SE PA

Re: Not enough a7 WUs

Postby bruce » Thu Mar 01, 2018 1:56 am

I do not know if the following is true, but it sounds like a plausible explanation.

Let's suppose that we start with about 10 000 WUs that can be assigned by the servers, say 7 000 are for A7 and 3 000 are for A4. You can reasonably assume that the average assign rate would be approximately 70% for A7. It should be noted that 10000 WUs is a small enough number that there will be times when getting ANY assignment might be a challenge so FAH needs to bring more projects on-line.

The process of bringing new projects on-line is based on the needs of scientific research. FAH does not issue "busy work" without genuine scientific value, just to keep the donors happy.

Now suppose that several new A4 projects are brought on-line with, say 1 200 000 new WUs. That gives us a condition where there are 1 203 000 WUs for a4 and still only 7 000 for A7. We can probably expect that 99% of the new assignments will be for A4 and less than 1% will be for A7. That might lead you to conclude (falsely) that there are "not enough A7 WUs."
bruce
 
Posts: 21543
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: Not enough a7 WUs

Postby DrBB1 » Thu Mar 01, 2018 9:17 am

bruce-- I'm convinced. Seems like the "normal course of business." One way Stanford could confirm your hypothesis—not saying Stanford should allocate resources to do this, but it could—would be to take all CPUs folding SMP/CPU WUs over, say, the past three months, and see if the distribution of the proportion of A4s assigned to each CPU is consistent with random assignment of available WUs. It's probably not that simple to do, anyway, since different CPUs fold at different rates, depending on their architecture and the WUs they fold, but given the large numbers of WUs and active CPUs over a significant period of time, a lot of that variation would cancel out. If someone actually did the study, hopefully no unanticipated bias in the assignment algorithm would be found. But if the study raised a red flag, I presume this would be a good thing for the Stanford folks to know. Again, given finite resources, I'm not recommending the Stanford folks actually spend time doing a study like the one I described, only that if anyone did, it would help to confirm your very reasonable hypothesis. :wink:
DrBB1
 
Posts: 159
Joined: Wed Mar 26, 2008 12:30 am
Location: SE PA

Re: Not enough a7 WUs

Postby parkut » Fri Mar 02, 2018 5:40 pm

At this moment, all 34 systems I manage have been assigned a7 wu's. Not a single a4. Very unusual.
User avatar
parkut
 
Posts: 410
Joined: Tue Feb 12, 2008 7:33 am
Location: SE Michigan, USA

Re: Not enough a7 WUs

Postby parkut » Fri Mar 02, 2018 9:53 pm

and 4 hours later, the mix has changed. Now have only 11 running a7 and 23 running a4 wu's
User avatar
parkut
 
Posts: 410
Joined: Tue Feb 12, 2008 7:33 am
Location: SE Michigan, USA

Re: Not enough a7 WUs

Postby Prettz » Thu Mar 29, 2018 2:34 pm

Despite the fact that they generate double the science, I get an a7 unit extremely rarely. Maybe one every few days? Why would the system do that? Why are new projects being made on a4 if it's outdated and inferior?
Prettz
 
Posts: 28
Joined: Sat Oct 17, 2009 6:47 pm

Re: Not enough a7 WUs

Postby JimboPalmer » Thu Mar 29, 2018 4:03 pm

Prettz wrote:Why are new projects being made on a4 if it's outdated and inferior?

The researcher (correctly) does not want to throw out all the results he/she has already collected on that molecule. Continuations of previous work almost always use the same core.
Tsar of all the Rushers
I tried to remain childlike, all I achieved was childish.
A friend to those who want no friends
JimboPalmer
 
Posts: 652
Joined: Mon Feb 16, 2009 4:12 am
Location: Greenwood MS USA

Re: Not enough a7 WUs

Postby DrBB1 » Fri Mar 30, 2018 2:59 am

Perhaps part of the discontent among some of us folders is that we really don't understand how WUs are assigned. For example, according to the Project Summary link at the top of the page, 45 WUs use the GRO_a4 core (about 55%), while 36 use the GRO_a7 core (about 45%). Yet, in the past week I have received only five a7 WUs out of a total of 47 (about 10%). This suggests at least the following three possibilities:

1. Not all "Active Projects" are really active at any given moment, and that currently almost all SMPs are receiving a small subset of the active projects, comprising mostly a4 projects.
2. All "Active Projects" are, indeed active, but the researchers are intentionally funneling the selection (either manually or by algorithm) toward greater selection of a4 WUs for reasonable purposes unknown to most of us folders (and most of us wouldn't be interested in the details, anyway).
3. All projects are active, but the selection algorithm has gone awry.

The "truth" may not be included among those three, and there may be other reasonable possibilities. However, a simple explanation on the FAH Blog about how lead scientists may select projects and WUs could end some apparent frustration among folders. [Of course, if #3 above explains the issue, the programmers have an out-of-control issue to deal with quickly, but I'm sure that someone would have noticed by now if that were the issue.] :D
DrBB1
 
Posts: 159
Joined: Wed Mar 26, 2008 12:30 am
Location: SE PA

Re: Not enough a7 WUs

Postby kiore » Fri Mar 30, 2018 4:35 am

DrBB1 wrote:Perhaps part of the discontent among some of us folders is that we really don't understand how WUs are assigned. For example, according to the Project Summary link at the top of the page, 45 WUs use the GRO_a4 core (about 55%), while 36 use the GRO_a7 core (about 45%). Yet, in the past week I have received only five a7 WUs out of a total of 47 (about 10%). This suggests at least the following three possibilities:

1. Not all "Active Projects" are really active at any given moment, and that currently almost all SMPs are receiving a small subset of the active projects, comprising mostly a4 projects.
2. All "Active Projects" are, indeed active, but the researchers are intentionally funneling the selection (either manually or by algorithm) toward greater selection of a4 WUs for reasonable purposes unknown to most of us folders (and most of us wouldn't be interested in the details, anyway).
3. All projects are active, but the selection algorithm has gone awry.

The "truth" may not be included among those three, and there may be other reasonable possibilities. However, a simple explanation on the FAH Blog about how lead scientists may select projects and WUs could end some apparent frustration among folders. [Of course, if #3 above explains the issue, the programmers have an out-of-control issue to deal with quickly, but I'm sure that someone would have noticed by now if that were the issue.] :D


Another option may be in play, in the past 'undesirable' work units have been dumped by folders in the hope of gaining more desirable ones, thus leading to them having to be redone and distorting the allocation. The quick return bonus penalty for failing to maintain an 80% completion rate has addressed this to some extent, but it remains possible that contributors may be distorting the project allocations this way.
Image

GTX 1080ti and GTX 1080 , OS= win7.
kiore
 
Posts: 958
Joined: Fri Jan 16, 2009 5:45 pm
Location: USA

Re: Not enough a7 WUs

Postby DrBB1 » Fri Mar 30, 2018 4:43 am

Another option may be in play, in the past 'undesirable' work units have been dumped by folders in the hope of gaining more desirable ones, thus leading to them having to be redone and distorting the allocation. The quick return bonus penalty for failing to maintain an 80% completion rate has addressed this to some extent, but it remains possible that contributors may be distorting the project allocations this way.


Of course, we could continue to speculate for many moons, but a simple explanation on the FAH Blog by some who actually knows, with a link to it in this discussion thread, would probably be far more satisfying to those who have expressed anger, frustration, and/or confusion about the seemingly inequitable distribution of a4 and a7 work units.
DrBB1
 
Posts: 159
Joined: Wed Mar 26, 2008 12:30 am
Location: SE PA

Re: Not enough a7 WUs

Postby bruce » Fri Mar 30, 2018 8:17 am

Each project is managed by someone in the scientific community. The number of active trajectories varies based on the needs of the research. Projects may be (and often are) suspended while data are being reduced, conclusions drawn, papers written, etc. or resumed when the intermediate results and new theories suggest a need for more simulation.

A trajectory is identified by Project, Run, Clone. with a number of segments identified by Generation number. Once Gen N is assigned, that trajectory waits until that gen is returned, at which time Gen (N+1) is prepared for assignment.

Many servers have WUs for a single FAHCore ... others have more than one type ... but as I said above, not all projects will necessarily be active. Unfortunately, I don't know any way to determine which projects are active, so there's no way to estimate the probability of getting an assignment for a particular FAHCore.

At this moment, there are 10 servers with WUs that can be assigned and assign-rates are currently as high as 2900. Actually, many of those servers have very few WUs -- probably waiting for somebody to return a WU that has been assigned so it can generate one more WU which can then be assignment. The bulk of the work is coming from four active servers at stanford.edu and at temple.edu.
bruce
 
Posts: 21543
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: Not enough a7 WUs

Postby DrBB1 » Fri Mar 30, 2018 7:41 pm

bruce, if I understand correctly, your explanation sounds like it may be most similar to my #1: "Not all 'Active Projects' are really active at any given moment" [and that—currently, anyway—almost all SMPs may be receiving just a subset of the "active projects," that just happens to comprise mostly a4 WUs]. I still think it would be useful from both and educative and customer service perspective if someone from the Pande Lab would post about the process of distributing WUs to folders in general, not just those of us who happen to see this discussion. [OK, I've said that three times now; 'nuf said!]
DrBB1
 
Posts: 159
Joined: Wed Mar 26, 2008 12:30 am
Location: SE PA

Re: Not enough a7 WUs

Postby JimboPalmer » Fri Mar 30, 2018 9:48 pm

You do realize that the folks at Pande Labs are Bio-Physicists, not computer techs, right?
JimboPalmer
 
Posts: 652
Joined: Mon Feb 16, 2009 4:12 am
Location: Greenwood MS USA

PreviousNext

Return to CPU Projects - released FAHCores _a4 & _a7

Who is online

Users browsing this forum: No registered users and 1 guest

cron