Repeated units / projects

[arbetus]

I would like ask official Stanford folding team for reason why we re-compute already finished units. For example two days ago I've computed for FOURTH time Project 10505 (Run 79, Clone 16, Gen 3). First time I received it on middle of October. Please note that I invested a lot of money to my computing capabilities (my average PPD is now > 160 000, I’m running 6 fully dedicated PCs) and I hope that my investment will help in scientific progress. Instead of that I noticed that I’m re-computing again and again the same units.
I'm starting to have serious doubts about the Folding project. I suspect that the Stanford team has not prepared new projects and therefore leaving again re-compute already completed tasks to prevent the departure of investigators. I am very concerned about the status thus wasting the resources invested voluntarily. If I not got a reasonable explanation, I would leave with the other members of our team from the Folding project.

Many thanks for your explanation in advance

arbetus (Marian Arbet, Pilsen, Czech Republic)

[7im]

Hello arbetus, welcome to the folding forum.

As a general policy, Stanford sends out a work unit only one time if that work unit is completed sucessfully and returned to Stanford successfully before the deadline expires. However, there are some rare circumstances when a work unit is not completed successfully, and the work unit is reassigned to another person to be completed. But once completed, the work unit does not go out again.

Also as a general rule, Stanford DOES NOT send out the same work unit multiple times to keep clients "busy" like some other infamous projects do. There is also an official statement about not sending out duplicate work units to keep clients "busy" in the Main FAQ: http://folding.stanford.edu/English/FAQ-main#ntoc14

However, in your case, I see that particular work unit has been sent out many many times. I also see several reports of problems with the project when I search this folding forum. This appears to be limited to a specific project # and/or and Assignment Server problem. It also looks to have been corrected recently. But it takes a little time for the duplicates to filter out of the system.

Additionally, to end your suspicion about a lack of new projects, please read Vijay's Updates thread in this forum. http://foldingforum.org/viewtopic.php?f=24&t=14714 New projects are being released and posted to that topic on a regular basis. And there is no lack of new project numbers appearing on the Project Summary page: http://fah-web.stanford.edu/psummary.html

I hope that addresses all of your concerns.

[gwildperson]

When was the last time you processed that WU?

I do see a problem which the Pande Group needs to fix, but it might be unrelated. Psummary says that it has zero atoms.

[arbetus]

Hello 7im, gwildperson,

many thanks for quick response. It seems that the issue with the Assigment Server continues - here is example for work unit Project 10505 (Run 113, Clone 36, Gen 3). I received it on one of my machines yesterday 31.12.2010 4:37 (UTC) and successfully completed it 31.12.2010 6:07 (UTC). I received the same unit on my another machine today a hour ago... and Psummary still says it has 0 atoms... Could you please check that ?

[bollix47]

From the project description page:

Since these are fragment simulations, each fragment has a slightly different number of atoms, expect some variability. These WUs were benchmarked using the fragment with the largest number of atoms. Any speedup you may observe for smaller fragments can be considered a bonus!

I suspect the variable number of atoms is why there is a zero on the summary page.

No idea why you're getting the exact same WU more than once but it's not unprecedented. We used to have projects that used core 14 where it was normal for the exact same WU identifier to be rerun.

viewtopic.php?f=52&t=13542&start=0

Are you getting credit for the duplicates? If so, this may be a similar situation to the one above.

[arbetus]

Hello bollix47,

how can I please find if I received credits for a particullar work unit ? I'm running a lot of clients therefore I'm unable to determine within the whole credit ammount if I received credits for one unit or not...

Here is another actual sample of terrible Assigment Server job:

[P5-133XL]

When I see one person with repeated duplicate WU's, I have to ask them to check to see if those specific clients are not using the same -gpu #. If is easy for a person to make that error and then the clients end up doing the same WU's for the server sees them as the same client.

[arbetus]

Hello P5-133XL,

no, that's not that case. I'm running most clients on machines with two GTX295 installed. That's 4 GPU clients per PC. I've checked very carefully and several times, that each client has it's unique client ID and is associated to different GPU. Also if there would be several clients sharing the same chip, the PPD of these clients would be lower. I'm runnning on good PPD level adequate to GTX295 performance...

[gwildperson]

It's not the GPU number that matters. It's the Machine ID which must be different on every client. Check the first page of FAHlog near where it prints your User Name and Team Number and make sure they're all unique within each copy of Windows that you run. (They can be duplicated on each separate machine since Stanford will assign each machine a unique User ID (not the same as User Name)

[arbetus]

gwildperson,

yes, they are setup correctly and have unique Machine ID's. I double checked that right now one more... ... btw HFM.NET is able detect and highlight clients with duplicated Machine ID... as you can see on the screenshot this check is OK for all monitored clients.

[PantherX]

For P5769 R8 C98 G298, I noticed that they were downloaded at the same time 5.1.2011 20:13 and I have read reports where if two same Clients ask for a WU at the same time, they will be assigned the same WU. I think that it might be a bug which I hope will be fixed.

If you get duplicate WUs and don't want to process them both (you will still get points for both of them if processed and return sucessfully), have you considered dumping 1 copy since the work will already be completed by the other copy?