Is Folding -bigadv on less than 8 threads cheating? [Yes]

The most demanding Projects are only available to a small percentage of very high-end servers.

Moderators: Site Moderators, PandeGroup

Re: Is Folding -bigadv on less than 8 threads cheating? [Yes]

Postby thefreeaccount » Wed Sep 08, 2010 11:04 pm

bruce wrote:
lanbrown wrote:How is the "science" being hurt by running them on those CPU's though?
You've already touched on a critical distinction. When there are lots of WUs, more CPUs, even if they're really close to the deadline is a good thing. When there is a shortage of WUs, it's important to assign them to the machines that will return them in the fastest possible time. In both cases, marginal performance drags down the averate turn-around time.

It seems to me that you have some very good points that are, unfortunately, being overlooked because they are wrapped up in a very poor overall message. A neutral/unbiased statement along the lines of 'There is a shortage of bigadv WUs. If you cannot do a pXXXXXin YY hours, please refrain from folding bigadv until the system has been adjusted' would not have generated five pages of responses.

I just cannot understand why you and several others keep refocusing the discussion on the number of physical, or worse yet, virtual cores a machine might have. What matters is overall system performance, which is best expressed in terms of time to complete a known set of work units. Number of cores has as much relevance to this discussion as the presence of a rear spoiler or a neon yellow paint stripe does to a discussion of whether a car is fast enough to enter an F1 grand prix.

The random swipes at people who overclock or use watercooling are not only unwarranted, but also hypocritical, given the large number of overclocked SR-2s currently folding bigadv.
thefreeaccount
 
Posts: 6
Joined: Wed Sep 08, 2010 10:19 pm

Re: Is Folding -bigadv on less than 8 threads cheating? [Yes]

Postby tear » Wed Sep 08, 2010 11:14 pm

thefreeaccount wrote:Number of cores has as much relevance to this discussion as the presence of a rear spoiler or a neon yellow paint stripe does to a discussion of whether a car is fast enough to enter an F1 grand prix.

Excellent point (for an observer).

Per your simile, race managers (of this race) do check for the rear spoiler and neon yellow paint stripes before allowing vehicle to enter one of race classes.
One man's ceiling is another man's floor.
Image
tear
 
Posts: 918
Joined: Sun Dec 02, 2007 4:08 am
Location: Rocky Mountains

Re: Is Folding -bigadv on less than 8 threads cheating? [Yes]

Postby rbpeake » Wed Sep 08, 2010 11:47 pm

thefreeaccount wrote:It seems to me that you have some very good points that are, unfortunately, being overlooked because they are wrapped up in a very poor overall message. A neutral/unbiased statement along the lines of 'There is a shortage of bigadv WUs. If you cannot do a pXXXXXin YY hours, please refrain from folding bigadv until the system has been adjusted' would not have generated five pages of responses.

If I understand your point, if there is a limited number of workunits available, let the fastest machines take them on. That way all the limited work available gets completed in the shortest possible time.

However, and here is where it gets interesting imho, if there are plenty of workunits, bring on more crunchers even if they are slower, because the alternative is that if just the fastest guys do the work, there is a limited number of "fastest guys" and the science overall will be advanced more slowly because there is a backlog of workunits that are just not getting done because the "fastest guys" simply cannot handle all the volume.

In this case it makes sense for "slower guys" to come aboard because even though they are slower, there are a lot of "slower guys" and overall more workunits will be completed in any unit of time, and thus the science progresses faster.

Does this logic make sense?
rbpeake
 
Posts: 325
Joined: Sun Jun 15, 2008 4:39 pm
Location: NYC Metro Area

Re: Is Folding -bigadv on less than 8 threads cheating? [Yes]

Postby 7im » Thu Sep 09, 2010 12:17 am

The logic makes sense up to a certain point. But then we are back at the Hyperthreading debate again. Would it be better to let a true 8 core machine guy fold a WU every 2 days by himself, or let a 2nd slower 4 day guy jump in?

The 4 day guy holds up the next WU an extra 2 days for all of the 2 day guys, slowing the average rate of return. The answer that we don't know is how long that WU in question would sit unprocessed without the 4 day guy jumping in. If it sits less than 2 days before a 2 day guy grabs it, then it's better to let the 2 day guys fold. If the WUs sit in the queue for longer than 2 days, then the 4 day guy jumping in is a net positive. You not only have to look at the average rate of return, but the opportunity cost of letting that WU sit a while so a faster guy can fold it, or letting the slower guys jump in.

And NONE of us have enough data to answer that question decisively. Only Stanford knows the queue times and rates of return, target and actual. To assume you know better than Stanford about this question is rather conceded.

Stanford sets out the rules and recommendations, and for the sake of the project, it's best we abide by them, not second guess them, not hack them, etc. You are more than welcome to suggest and lobby for changes to those policies. Presenting hard facts, like actual x6 system performance numbers, fahlogs, frametimes, etc. would be a good first step to providing good reasons for revising their 8 core policy.

Continuing to debate what the original policy actual means is futile.
How to provide enough information to get helpful support
Tell me and I forget. Teach me and I remember. Involve me and I learn.
User avatar
7im
 
Posts: 15237
Joined: Thu Nov 29, 2007 4:30 pm
Location: Arizona

Re: Is Folding -bigadv on less than 8 threads cheating? [Yes]

Postby gwildperson » Thu Sep 09, 2010 1:25 am

thefreeaccount wrote:A neutral/unbiased statement along the lines of 'There is a shortage of bigadv WUs. If you cannot do a pXXXXX in YY hours, please refrain from folding bigadv until the system has been adjusted' would not have generated five pages of responses.


Such a statement would be valid (A) If somebody, somewhere, knew the value of YY for every value of XXXXX and actively tracked YY as it changed, and (B1) if everybody was instantly voluntarily configured their system to follow that recommendation, or (B2) something, somewhere, knew which clients would be able/not_able to meet the YY requirement and automatically caused the assignment server to issue/not_issue XXXXX to that client.

Since the current value of all YY's is not known, since people who want a high PPD don't voluntarily follow suggested recommendations if it means they'll get a lower PPD, and since performance prediction by individual clients isn't part of Stanford's current software, the developers have three (or more) unsolved problems, none of which are trivial.

The client does detect very few characteristics of the hardware in an effort to solve B2: It does know your RAM and it does know the number of cores your system reports. If that's the only data available, then they're forced to use it in an all-or-nothing decision process to assign what appear to be powerful systems to the XXXXX's with the most critical YY's.

Does a 24-core system fold faster than a 2-core system? Probably. OK. Establish some cutoff point and assign based exclusively on what you do know about that system. Stanford set cores>=8 as the cutoff. That's the limit of what the software can do.

But what about B1? Obviously at lease one whole team is so focused on PPD that they have decided it's not cheating if they provide fraudulent credentials to bypass the admittedly weak criteria being used by Stanford. Moreove they continue to justify that their fraudulent credentials are acceptable based on the fact that they can get away with it.

What matters is overall system performance, which is best expressed in terms of time to complete a known set of work units.
True. Let's see if we can apply method B1.

I'm going to make an arbitrary guess that YY=2 days. Would all those who cannot return a WU within 2 days please immediately stop trying to get a bigadv WU. If the number of WUs available continues to remain about the same, then it was a good guess or perhaps it should have been YY=1.5 days since we already have a shortage. If within a day or two we have an abundance of WUs not being processed, then we'll adjust it to be YY=2.5 and see what happens. Does that work for everybody?

(I think I hear the sounds of laughter . . . or maybe it's a rebellion starting somewhere . . . . );)

OK, :idea: since voluntary compliance won't work, maybe we can ask Stanford to add an incentive. Suppose we allow some objective person (not part of any team!) to adjust the today's preferred deadline from 4 days to 2 days and to adjust again any time it's appropriate. Everybody who starts a bigadv WU will still get baseline points if they exceed YY, right up to the 6 day Final deadline. If enough people don't voluntarily comply, a bunch of people will simply not get a bonus when they happen to violate today's recommended YY=2 (or whatever value it takes on tomorrow). Except for the extra labor required to check on the quantity of WUs available and to adjust YY dynamically, I'm sure that's a workable system. Would that make everybody happy?

To make this fair, we'll leave the bonus plan for standard SMP alone and only adjust the bonus cutoff for bigadv on a dynamic basis. It will be up to you to remove the -bigadv flag if it doesn't produce the desired results for you. You can choose between a guaranteed bonus for -smp or a wager that you might get a bigger bonus or none at all if you choose -bigadv.

(Now I know I really DO hear the sounds of a rebellion brewing.) :!:
gwildperson
 
Posts: 785
Joined: Tue Dec 04, 2007 8:36 pm

Re: Is Folding -bigadv on less than 8 threads cheating? [Yes]

Postby PantherX » Thu Sep 09, 2010 3:27 am

@gwildperson:
How would you explain to other folders when PDC comes back online and rips through bigadv WUs? I believe that it's too much of a hassle and involves a significant coding process.
Instead I am rooting for my own idea:
PantherX wrote:...Every time a WU is finished (regardless of the Client type), a performance fraction is updated to reflect the last 4 WUs done. The higher the PF is the faster the system folds so if a Client has -bigadv flag, the Server requests for the PF. If it is higher than a fixed value (set by PG) the Client get the bigadv WU. If it is lower, then the Server checks the bigadv WU availability. If it is sufficiently high, it assigns the bigadv WU. If there is a shortage, you get normal WU. I think that it will be a "perfect" solution for this i7 VS multi-socket system...
As you can see, "half" the work is done. The Client has the PF that is generated for the last 4 Slots (unless this is removed in v7). The other "half" involves the Server Codes being updated to recognized PF. Also if the logic of WU assignment is tweaked to give Clients with high PF preference over lower ones, everybody would be happy.

Do note that the "low-end" users (mostly) agree that if there is a shortage of bigadv, priority should be for multi-socket systems and in abundance, they can continue to fold bigadv WUs. This also takes into consideration 7im's point about feasibility of "low-end" users. This may be far from perfect but at least I believe that its a step towards a "happier" donor system.
ETA:
Now ↞ Very Soon ↔ Soon ↔ Soon-ish ↔ Not Soon ↠ End Of Time

Welcome To The F@H Support Forum Ӂ Chrome Folding App (Beta) Ӂ Troubleshooting "Bad WUs" Ӂ Troubleshooting Server Connectivity Issues
User avatar
PantherX
Site Moderator
 
Posts: 6614
Joined: Wed Dec 23, 2009 9:33 am

Re: Is Folding -bigadv on less than 8 threads cheating? [Yes]

Postby gwildperson » Thu Sep 09, 2010 3:51 am

My discussion of method B1 was sarcastic (and intended to be just as provocative as some of the other posts that I've seen). In simple terms, I do not believe that any voluntary method is going to work unless some serious changes are made to the bonus program.

A server-managed system such as the one you propose is much better. I don't think your proposal to use PF will work any better than the number-of-cores method. Both are open to hacking and I doubt that it would take very long for somebody to figure out how to present false PF credentials to the server and cheat that system just like question that originally started this topic (so even a non-voluntary system won't work if it depends on information coming from the client).

Either the bigadv bonus is too big (and reducing it might make cheating no longer worth the effort) or it has to be a hack-proof system ("sure" that's going to happen ;)).
gwildperson
 
Posts: 785
Joined: Tue Dec 04, 2007 8:36 pm

Re: Is Folding -bigadv on less than 8 threads cheating? [Yes]

Postby P5-133XL » Thu Sep 09, 2010 5:19 am

There's really nothing stopping Stanford from giving priority to machines that are very fast or filtering out people that are slow to return depending on the availability of WU's. They keep a unique number for every install as shown by the ability of the assignment servers to reissue the same WU if a client requests a new WU before returning the previous one. They also have the ability to time how long it takes a specific client to do a WU which is necessary to calculate the bonus points. They just need to keep a history database and they can create an accurate performance metric. All the tools are there for them to be able to implement a client independent performance based priority system if that was what they wanted to do.
Image
P5-133XL
 
Posts: 4345
Joined: Sun Dec 02, 2007 4:36 am
Location: Salem. OR USA

Re: Is Folding -bigadv on less than 8 threads cheating? [Yes]

Postby shdbcamping » Thu Sep 09, 2010 2:37 pm

This is all becoming a cumbersome argument where turnaround times per prcg is constantly alluded to. It's a matter of Statistics...

Since from what I have noticed, the PRCG's are not issued sequentially.... The fastest argument is NULL. Let's do some Math, shall we? As I understand it, the RCG's of a project are multiply assigned. There are Exponentially exploded 'possible' bottlenecks for every assignment by the server. All the 'really fast' systems could 'crash' leaving the 'slow machine in 1st.

That being said, multiple returns of a project (prcg) would be a good thing as there is a 'comparative' result for verification in the event that one result would appear 'out of the expected limits'. This IS a science 'thingy' and therefore the 1st in 'only' argument is inadequate in the realm of 'correctness' of results. I'm sure this is already built in to the F@H model. At least, I would hope so, as VJay seems more than 'competant' JIMO.

With those points in mind, why does it matter if a "slow" result comes in that would give PG reason to go back and 're-look' at a result that may later become suspect.... especially if it causes "false" results to be overturned?

'S'All I'm trying to throw out there'. This is a Science endeavor IIRC :wink:

Sean
shdbcamping
 
Posts: 587
Joined: Mon Nov 10, 2008 7:57 am

Re: Is Folding -bigadv on less than 8 threads cheating? [Yes]

Postby 7im » Thu Sep 09, 2010 4:16 pm

P5-133XL wrote:There's really nothing stopping Stanford from giving priority to machines that are very fast or filtering out people that are slow to return depending on the availability of WU's. They keep a unique number for every install as shown by the ability of the assignment servers to reissue the same WU if a client requests a new WU before returning the previous one. They also have the ability to time how long it takes a specific client to do a WU which is necessary to calculate the bonus points. They just need to keep a history database and they can create an accurate performance metric. All the tools are there for them to be able to implement a client independent performance based priority system if that was what they wanted to do.


Actually, there is a lot of things stopping Stanford. Time, money, programming requirements to keep a huge project running while trying to make feature improvements, backwards compatibility, actually having to do some science instead of play games with clients. :roll:

Stanford's first run at a two tiered bonus program is obviously rudimentary. And adding most of what you talked about for improving the bonus program is not easy or even possible. Like trying to run an ecommerce site like Amazon from a giant Excel spreadsheet as your database. The tools just aren't up to the task you're asking them to do.
User avatar
7im
 
Posts: 15237
Joined: Thu Nov 29, 2007 4:30 pm
Location: Arizona

Re: Is Folding -bigadv on less than 8 threads cheating? [Yes]

Postby 10e » Fri Sep 10, 2010 2:43 pm

PantherX wrote:@gwildperson:
How would you explain to other folders when PDC comes back online and rips through bigadv WUs? I believe that it's too much of a hassle and involves a significant coding process.
Instead I am rooting for my own idea:
PantherX wrote:...Every time a WU is finished (regardless of the Client type), a performance fraction is updated to reflect the last 4 WUs done. The higher the PF is the faster the system folds so if a Client has -bigadv flag, the Server requests for the PF. If it is higher than a fixed value (set by PG) the Client get the bigadv WU. If it is lower, then the Server checks the bigadv WU availability. If it is sufficiently high, it assigns the bigadv WU. If there is a shortage, you get normal WU. I think that it will be a "perfect" solution for this i7 VS multi-socket system...
As you can see, "half" the work is done. The Client has the PF that is generated for the last 4 Slots (unless this is removed in v7). The other "half" involves the Server Codes being updated to recognized PF. Also if the logic of WU assignment is tweaked to give Clients with high PF preference over lower ones, everybody would be happy.

Do note that the "low-end" users (mostly) agree that if there is a shortage of bigadv, priority should be for multi-socket systems and in abundance, they can continue to fold bigadv WUs. This also takes into consideration 7im's point about feasibility of "low-end" users. This may be far from perfect but at least I believe that its a step towards a "happier" donor system.


I am 99% in agreement with this, and if the logic can be adjusted without a herculean programming effort, I'd say it's a good idea.

My only reservation is in how the logic would apply to the back-end Stanford systems, of which I am obviously ignorant. If people who had BigAdv capable machines that were below said PF asked for BigAdvs but kept getting A3s, and the BigAdvs "sat in a queue" waiting for a high PF machine, the project would not be properly served and this could be counterproductive. I don't know if there would ever be a case where the PF "cutoff" could be lowered or adjusted when there is a BigAdv "surplus", which doesn't seem to be happening much lately :biggrin:

It's a tough issue, and that's not taking the donors' feelings about being above or below said PF cutoff, possibly leading to both client and donor unexpected behavior.
Image Image
10e
 
Posts: 83
Joined: Thu Mar 05, 2009 12:36 pm
Location: Toronto, Ontario, CANADA

Re: Is Folding -bigadv on less than 8 threads cheating? [Yes]

Postby kasson » Mon Sep 13, 2010 4:16 pm

A clarification has been requested here. The purpose of bigadv is to run specific systems that are often large and slow and where we typically want a few relatively long trajectories rather than many shorter ones. As a result, the number of work units is often (though not always) limited, and we want to incentivize the fastest turnaround possible.

Bigadv was really designed for systems with >=8 cores. Manipulating the system to run bigadv on systems with 6 physical cores is possible; we would discourage it but aren't at the moment actively declaring it cheating if done without modifying the client. HOWEVER, we really want to emphasize the design for systems with >=8 cores. Just because someone can make the deadlines on current bigadv projects with certain 6-core systems is no guarantee that it will be possible in the future. If you're trying to game the system, complaining when the manipulation stops yielding you results is bad form. We'd really rather encourage more people to run bigadv with 12, 24, 48, etc. cores.

As mentioned above, there are a lot of more sophisticated things we could do for allocating machines and WU's. We'd love to be able to do them; some have technical issues, and in general we're trying to run a robust system with limited programming resources. And our primary goal is to do good science, with the help of our donors. We have some resources dedicated to infrastructure, but a lot of other work involves taking people away from science to work on infrastructure.
User avatar
kasson
Pande Group Member
 
Posts: 1909
Joined: Thu Nov 29, 2007 9:37 pm

Previous

Return to SMP with bigadv

Who is online

Users browsing this forum: No registered users and 1 guest

cron