13405 (Run 597, Clone 71, Gen 0)

Moderators: Site Moderators, FAHC Science Team

13405 (Run 597, Clone 71, Gen 0)

Postby SAK917 » Wed May 13, 2020 9:28 pm

System Info:
Code: Select all
*********************** Log Started 2020-05-11T18:09:06Z ***********************
18:09:06:Trying to access database...
18:09:06:Successfully acquired database lock
18:09:06:Downloading GPUs.txt from assign1.foldingathome.org:80
18:09:06:Connecting to assign1.foldingathome.org:80
18:09:06:Read GPUs.txt
18:09:07:Enabled folding slot 01: READY gpu:0:TU102 [GeForce RTX 2080 Ti Rev. A] M 13448
18:09:07:Enabled folding slot 00: READY cpu:8
18:09:07:****************************** FAHClient ******************************
18:09:07:        Version: 7.6.13
18:09:07:         Author: Joseph Coffland <joseph@cauldrondevelopment.com>
18:09:07:      Copyright: 2020 foldingathome.org
18:09:07:       Homepage: https://foldingathome.org/
18:09:07:           Date: Apr 27 2020
18:09:07:           Time: 21:21:01
18:09:07:       Revision: 5a652817f46116b6e135503af97f18e094414e3b
18:09:07:         Branch: master
18:09:07:       Compiler: Visual C++ 2008
18:09:07:        Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
18:09:07:       Platform: win32 10
18:09:07:           Bits: 32
18:09:07:           Mode: Release
18:09:07:         Config: E:\FAHClient\config.xml
18:09:07:******************************** CBang ********************************
18:09:07:           Date: Apr 24 2020
18:09:07:           Time: 17:07:55
18:09:07:       Revision: ea081a3b3b0f4a37c4d0440b4f1bc184197c7797
18:09:07:         Branch: master
18:09:07:       Compiler: Visual C++ 2008
18:09:07:        Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
18:09:07:       Platform: win32 10
18:09:07:           Bits: 32
18:09:07:           Mode: Release
18:09:07:******************************* System ********************************
18:09:07:            CPU: Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz
18:09:07:         CPU ID: GenuineIntel Family 6 Model 158 Stepping 13
18:09:07:           CPUs: 16
18:09:07:         Memory: 31.86GiB
18:09:07:    Free Memory: 26.96GiB
18:09:07:        Threads: WINDOWS_THREADS
18:09:07:     OS Version: 6.2
18:09:07:    Has Battery: true
18:09:07:     On Battery: false
18:09:07:     UTC Offset: -7
18:09:07:            PID: 16772
18:09:07:            CWD: E:\FAHClient
18:09:07:  Win32 Service: false
18:09:07:             OS: Windows 10 Enterprise
18:09:07:        OS Arch: AMD64
18:09:07:           GPUs: 1
18:09:07:          GPU 0: Bus:1 Slot:0 Func:0 NVIDIA:8 TU102 [GeForce RTX 2080 Ti Rev. A]
18:09:07:                 M 13448
18:09:07:  CUDA Device 0: Platform:0 Device:0 Bus:1 Slot:0 Compute:7.5 Driver:10.2
18:09:07:OpenCL Device 0: Platform:0 Device:0 Bus:NA Slot:NA Compute:2.1 Driver:26.20
18:09:07:OpenCL Device 2: Platform:1 Device:0 Bus:1 Slot:0 Compute:1.2 Driver:442.92
18:09:07:******************************* libFAH ********************************
18:09:07:           Date: Apr 15 2020
18:09:07:           Time: 14:53:14
18:09:07:       Revision: 216968bc7025029c841ed6e36e81a03a316890d3
18:09:07:         Branch: master
18:09:07:       Compiler: Visual C++ 2008
18:09:07:        Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
18:09:07:       Platform: win32 10
18:09:07:           Bits: 32
18:09:07:           Mode: Release
18:09:07:***********************************************************************
18:09:07:<config>
18:09:07:  <!-- Folding Core -->
18:09:07:  <checkpoint v='20'/>
18:09:07:
18:09:07:  <!-- Folding Slot Configuration -->
18:09:07:  <client-type v='advanced'/>
18:09:07:
18:09:07:  <!-- Network -->
18:09:07:  <proxy v=':8080'/>
18:09:07:
18:09:07:  <!-- Slot Control -->
18:09:07:  <power v='medium'/>
18:09:07:
18:09:07:  <!-- User Information -->
18:09:07:  <passkey v='*****'/>
18:09:07:  <team v='**'/>
18:09:07:  <user v='**'/>
18:09:07:
18:09:07:  <!-- Folding Slots -->
18:09:07:  <slot id='1' type='GPU'/>
18:09:07:  <slot id='0' type='CPU'>
18:09:07:    <cpus v='8'/>
18:09:07:  </slot>
18:09:07:</config>


WU Processing Log:
Code: Select all
******************************* Date: 2020-05-13 *******************************
00:28:35:WU01:FS01:Connecting to assign1.foldingathome.org:80
00:28:35:WARNING:WU01:FS01:Failed to get assignment from 'assign1.foldingathome.org:80': No WUs available for this configuration
00:28:35:WU01:FS01:Connecting to assign2.foldingathome.org:80
00:28:35:WARNING:WU01:FS01:Failed to get assignment from 'assign2.foldingathome.org:80': No WUs available for this configuration
00:28:35:WU01:FS01:Connecting to assign3.foldingathome.org:80
00:28:36:WU01:FS01:Assigned to work server 3.133.76.19
00:28:36:WU01:FS01:Requesting new work unit for slot 01: RUNNING gpu:0:TU102 [GeForce RTX 2080 Ti Rev. A] M 13448 from 3.133.76.19
00:28:36:WU01:FS01:Connecting to 3.133.76.19:8080
00:28:57:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
00:28:57:WU01:FS01:Connecting to 3.133.76.19:80
00:30:08:ERROR:WU01:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
00:30:08:WU01:FS01:Connecting to assign1.foldingathome.org:80
00:30:08:WU01:FS01:Assigned to work server 18.188.125.154
00:30:08:WU01:FS01:Requesting new work unit for slot 01: READY gpu:0:TU102 [GeForce RTX 2080 Ti Rev. A] M 13448 from 18.188.125.154
00:30:08:WU01:FS01:Connecting to 18.188.125.154:8080
00:30:10:WU01:FS01:Downloading 6.36MiB
00:30:12:WU01:FS01:Download complete
00:30:12:WU01:FS01:Received Unit: id:01 state:DOWNLOAD error:NO_ERROR project:13405 run:597 clone:71 gen:0 core:0x22 unit:0x0000000412bc7d9a5eb97d3d72f48c4e
00:30:12:WU01:FS01:Starting
00:30:12:WU01:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" E:\FAHClient\cores/cores.foldingathome.org/v7/win/64bit/Core_22.fah/FahCore_22.exe -dir 01 -suffix 01 -version 706 -lifeline 16772 -checkpoint 20 -gpu-vendor nvidia -opencl-platform 1 -opencl-device 0 -cuda-device 0 -gpu 0
00:30:12:WU01:FS01:Started FahCore on PID 1700
00:30:12:WU01:FS01:Core PID:17512
00:30:12:WU01:FS01:FahCore 0x22 started
00:30:12:WU01:FS01:0x22:*********************** Log Started 2020-05-13T00:30:12Z ***********************
00:30:12:WU01:FS01:0x22:*************************** Core22 Folding@home Core ***************************
00:30:12:WU01:FS01:0x22:       Type: 0x22
00:30:12:WU01:FS01:0x22:       Core: Core22
00:30:12:WU01:FS01:0x22:    Website: https://foldingathome.org/
00:30:12:WU01:FS01:0x22:  Copyright: (c) 2009-2018 foldingathome.org
00:30:12:WU01:FS01:0x22:     Author: John Chodera <john.chodera@choderalab.org> and Rafal Wiewiora
00:30:12:WU01:FS01:0x22:             <rafal.wiewiora@choderalab.org>
00:30:12:WU01:FS01:0x22:       Args: -dir 01 -suffix 01 -version 706 -lifeline 1700 -checkpoint 20
00:30:12:WU01:FS01:0x22:             -gpu-vendor nvidia -opencl-platform 1 -opencl-device 0 -cuda-device
00:30:12:WU01:FS01:0x22:             0 -gpu 0
00:30:12:WU01:FS01:0x22:     Config: <none>
00:30:12:WU01:FS01:0x22:************************************ Build *************************************
00:30:12:WU01:FS01:0x22:    Version: 0.0.5
00:30:12:WU01:FS01:0x22:       Date: Apr 22 2020
00:30:12:WU01:FS01:0x22:       Time: 04:42:59
00:30:12:WU01:FS01:0x22: Repository: Git
00:30:12:WU01:FS01:0x22:   Revision: 2d69202c898bd9bb3e093f51cd32bf411c2a0388
00:30:12:WU01:FS01:0x22:     Branch: HEAD
00:30:12:WU01:FS01:0x22:   Compiler: Visual C++ 2008
00:30:12:WU01:FS01:0x22:    Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
00:30:12:WU01:FS01:0x22:   Platform: win32 10
00:30:12:WU01:FS01:0x22:       Bits: 64
00:30:12:WU01:FS01:0x22:       Mode: Release
00:30:12:WU01:FS01:0x22:************************************ System ************************************
00:30:12:WU01:FS01:0x22:        CPU: Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz
00:30:12:WU01:FS01:0x22:     CPU ID: GenuineIntel Family 6 Model 158 Stepping 13
00:30:12:WU01:FS01:0x22:       CPUs: 16
00:30:12:WU01:FS01:0x22:     Memory: 31.86GiB
00:30:12:WU01:FS01:0x22:Free Memory: 20.54GiB
00:30:12:WU01:FS01:0x22:    Threads: WINDOWS_THREADS
00:30:12:WU01:FS01:0x22: OS Version: 6.2
00:30:12:WU01:FS01:0x22:Has Battery: true
00:30:12:WU01:FS01:0x22: On Battery: false
00:30:12:WU01:FS01:0x22: UTC Offset: -7
00:30:12:WU01:FS01:0x22:        PID: 17512
00:30:12:WU01:FS01:0x22:        CWD: E:\FAHClient\work
00:30:12:WU01:FS01:0x22:         OS: Windows 10 Pro
00:30:12:WU01:FS01:0x22:    OS Arch: AMD64
00:30:12:WU01:FS01:0x22:********************************************************************************
00:30:12:WU01:FS01:0x22:Project: 13405 (Run 597, Clone 71, Gen 0)
00:30:12:WU01:FS01:0x22:Unit: 0x0000000412bc7d9a5eb97d3d72f48c4e
00:30:12:WU01:FS01:0x22:Reading tar file core.xml
00:30:12:WU01:FS01:0x22:Reading tar file integrator.xml
00:30:12:WU01:FS01:0x22:Reading tar file state.xml
00:30:12:WU01:FS01:0x22:Reading tar file system.xml
00:30:12:WU01:FS01:0x22:Digital signatures verified
00:30:12:WU01:FS01:0x22:Folding@home GPU Core22 Folding@home Core
00:30:12:WU01:FS01:0x22:Version 0.0.5
00:30:19:WU01:FS01:0x22:Completed 0 out of 1000000 steps (0%)
00:30:19:WU01:FS01:0x22:Temperature control disabled. Requirements: single Nvidia GPU, tmax must be < 110 and twait >= 900
00:31:14:WU01:FS01:0x22:Completed 10000 out of 1000000 steps (1%)
00:32:08:WU01:FS01:0x22:Completed 20000 out of 1000000 steps (2%)
00:33:02:WU01:FS01:0x22:Completed 30000 out of 1000000 steps (3%)
00:33:57:WU01:FS01:0x22:Completed 40000 out of 1000000 steps (4%)
00:34:51:WU01:FS01:0x22:Completed 50000 out of 1000000 steps (5%)
00:35:45:WU01:FS01:0x22:Completed 60000 out of 1000000 steps (6%)
00:36:40:WU01:FS01:0x22:Completed 70000 out of 1000000 steps (7%)
00:37:34:WU01:FS01:0x22:Completed 80000 out of 1000000 steps (8%)
00:38:28:WU01:FS01:0x22:Completed 90000 out of 1000000 steps (9%)
00:39:23:WU01:FS01:0x22:Completed 100000 out of 1000000 steps (10%)
00:40:17:WU01:FS01:0x22:Completed 110000 out of 1000000 steps (11%)
00:41:12:WU01:FS01:0x22:Completed 120000 out of 1000000 steps (12%)
00:42:06:WU01:FS01:0x22:Completed 130000 out of 1000000 steps (13%)
00:43:00:WU01:FS01:0x22:Completed 140000 out of 1000000 steps (14%)
00:43:55:WU01:FS01:0x22:Completed 150000 out of 1000000 steps (15%)
00:44:49:WU01:FS01:0x22:Completed 160000 out of 1000000 steps (16%)
00:45:44:WU01:FS01:0x22:Completed 170000 out of 1000000 steps (17%)
00:46:38:WU01:FS01:0x22:Completed 180000 out of 1000000 steps (18%)
00:47:32:WU01:FS01:0x22:Completed 190000 out of 1000000 steps (19%)
00:48:27:WU01:FS01:0x22:Completed 200000 out of 1000000 steps (20%)
00:49:21:WU01:FS01:0x22:Completed 210000 out of 1000000 steps (21%)
00:50:15:WU01:FS01:0x22:Completed 220000 out of 1000000 steps (22%)
00:51:10:WU01:FS01:0x22:Completed 230000 out of 1000000 steps (23%)
00:52:04:WU01:FS01:0x22:Completed 240000 out of 1000000 steps (24%)
00:52:58:WU01:FS01:0x22:Completed 250000 out of 1000000 steps (25%)
00:54:00:WU01:FS01:0x22:Completed 260000 out of 1000000 steps (26%)
00:55:00:WU01:FS01:0x22:Completed 270000 out of 1000000 steps (27%)
00:56:00:WU01:FS01:0x22:Completed 280000 out of 1000000 steps (28%)
00:57:00:WU01:FS01:0x22:Completed 290000 out of 1000000 steps (29%)
00:58:00:WU01:FS01:0x22:Completed 300000 out of 1000000 steps (30%)
00:59:01:WU01:FS01:0x22:Completed 310000 out of 1000000 steps (31%)
01:00:01:WU01:FS01:0x22:Completed 320000 out of 1000000 steps (32%)
01:01:01:WU01:FS01:0x22:Completed 330000 out of 1000000 steps (33%)
01:02:01:WU01:FS01:0x22:Completed 340000 out of 1000000 steps (34%)
01:03:02:WU01:FS01:0x22:Completed 350000 out of 1000000 steps (35%)
01:04:02:WU01:FS01:0x22:Completed 360000 out of 1000000 steps (36%)
01:05:02:WU01:FS01:0x22:Completed 370000 out of 1000000 steps (37%)
01:06:02:WU01:FS01:0x22:Completed 380000 out of 1000000 steps (38%)
01:07:02:WU01:FS01:0x22:Completed 390000 out of 1000000 steps (39%)
01:08:02:WU01:FS01:0x22:Completed 400000 out of 1000000 steps (40%)
01:09:02:WU01:FS01:0x22:Completed 410000 out of 1000000 steps (41%)
01:10:03:WU01:FS01:0x22:Completed 420000 out of 1000000 steps (42%)
01:11:02:WU01:FS01:0x22:Completed 430000 out of 1000000 steps (43%)
01:12:03:WU01:FS01:0x22:Completed 440000 out of 1000000 steps (44%)
01:13:03:WU01:FS01:0x22:Completed 450000 out of 1000000 steps (45%)
01:14:04:WU01:FS01:0x22:Completed 460000 out of 1000000 steps (46%)
01:15:04:WU01:FS01:0x22:Completed 470000 out of 1000000 steps (47%)
01:16:04:WU01:FS01:0x22:Completed 480000 out of 1000000 steps (48%)
01:17:05:WU01:FS01:0x22:Completed 490000 out of 1000000 steps (49%)
01:18:08:WU01:FS01:0x22:Bad State detected... attempting to resume from last good checkpoint. Is your system overclocked?
01:18:08:WU01:FS01:0x22:Following exception occured: Particle coordinate is nan
01:18:19:WU01:FS01:0x22:Bad State detected... attempting to resume from last good checkpoint. Is your system overclocked?
01:18:19:WU01:FS01:0x22:Following exception occured: Particle coordinate is nan
01:18:29:WU01:FS01:0x22:Bad State detected... attempting to resume from last good checkpoint. Is your system overclocked?
01:18:29:WU01:FS01:0x22:Following exception occured: Particle coordinate is nan
01:18:29:WU01:FS01:0x22:ERROR:114: Max Retries Reached
01:18:29:WU01:FS01:0x22:Saving result file ..\logfile_01.txt
01:18:29:WU01:FS01:0x22:Saving result file badstate-0.xml
01:18:29:WU01:FS01:0x22:Saving result file badstate-1.xml
01:18:29:WU01:FS01:0x22:Saving result file badstate-2.xml
01:18:29:WU01:FS01:0x22:Saving result file checkpointState.xml
01:18:30:WU01:FS01:0x22:Saving result file checkpt.crc
01:18:30:WU01:FS01:0x22:Saving result file globals.csv
01:18:30:WU01:FS01:0x22:Saving result file positions.xtc
01:18:30:WU01:FS01:0x22:Saving result file science.log
01:18:30:WU01:FS01:0x22:Folding@home Core Shutdown: BAD_WORK_UNIT
01:18:31:WARNING:WU01:FS01:FahCore returned: BAD_WORK_UNIT (114 = 0x72)
01:18:31:WU01:FS01:Sending unit results: id:01 state:SEND error:FAULTY project:13405 run:597 clone:71 gen:0 core:0x22 unit:0x0000000412bc7d9a5eb97d3d72f48c4e
01:18:31:WU01:FS01:Uploading 4.97MiB to 18.188.125.154
01:18:31:WU01:FS01:Connecting to 18.188.125.154:8080
01:18:37:WU01:FS01:Upload 71.61%
01:18:39:WU01:FS01:Upload complete
01:18:39:WU01:FS01:Server responded WORK_ACK (400)
01:18:39:WU01:FS01:Cleaning up


Science.log file appears to be lost as the system moved on to process a new WU. Is there a way to have it save the science.log file for failed WUs?
SAK917
 
Posts: 10
Joined: Tue May 12, 2020 5:17 pm

Re: 13405 (Run 597, Clone 71, Gen 0)

Postby PantherX » Thu May 14, 2020 5:00 am

Unfortunately, there's no way to save the science.log file of the WU automatically AFAIK. You will have to save it before the clean-up happens :/
ETA:
Now ↞ Very Soon ↔ Soon ↔ Soon-ish ↔ Not Soon ↠ End Of Time

Welcome To The F@H Support Forum Ӂ Troubleshooting Bad WUs Ӂ Troubleshooting Server Connectivity Issues
User avatar
PantherX
Site Moderator
 
Posts: 6850
Joined: Wed Dec 23, 2009 10:33 am
Location: Land Of The Long White Cloud

Re: 13405 (Run 597, Clone 71, Gen 0)

Postby bruce » Thu May 14, 2020 5:10 am

The science log does get uploaded to the scientist who owns the project. (That's its primary function.)
bruce
 
Posts: 20118
Joined: Thu Nov 29, 2007 11:13 pm
Location: So. Cal.

Re: 13405 (Run 597, Clone 71, Gen 0)

Postby SAK917 » Thu May 14, 2020 5:59 am

That's good since I never seem to catch the WU right when it fails. If it gets to the scientist one way or another then I won't worry about it... thanks for clarifying that Bruce.
SAK917
 
Posts: 10
Joined: Tue May 12, 2020 5:17 pm

Re: 13405 (Run 597, Clone 71, Gen 0)

Postby bruce » Thu May 14, 2020 6:10 am

I'm confident that Dr Chodera will sort it out and make whatever changes he thinks will help with the next batch of WU.
bruce
 
Posts: 20118
Joined: Thu Nov 29, 2007 11:13 pm
Location: So. Cal.


Return to Issues with a specific WU

Who is online

Users browsing this forum: Yandex [Bot] and 1 guest

cron