Project 13414

Moderators: Site Moderators, FAHC Science Team

Curt3g
Posts: 16
Joined: Sun Mar 29, 2020 4:27 pm

Project 13414

Post by Curt3g »

Following up on a post from this morning that I may not have put in the right forum (leave it to a new guy :D ).

viewtopic.php?f=74&t=35624&start=15#p338142 (includes log extracts)

The final processing time on Project: 13414 (Run 824, Clone 39, Gen 1) was a little over 13 hours (RTX 2060).

Not experienced enough to know if this is an issue or not.

Cheers,
Curt
bruce
Posts: 20910
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: Project 13414

Post by bruce »

The preferred deadline is 1.0 days and the final deadline is 2.0 days for that project. 13 hours isn't unreasonable.
Curt3g
Posts: 16
Joined: Sun Mar 29, 2020 4:27 pm

Re: Project 13414

Post by Curt3g »

Project: 13412 (Run 2439, Clone 10, Gen 0) was processed a little bit later last night, with a run time a hair over 4 hrs.

Cheers,
JohnChodera
Pande Group Member
Posts: 470
Joined: Fri Feb 22, 2013 9:59 pm

Re: Project 13414

Post by JohnChodera »

Wow, we were very much aiming for ~4h, so 13h sounds like something weird is going on. We're seeing some heterogeneity in our calculation, and are looking into what is causing this.

Thanks so much!

~ John Chodera // MSKCC
Curt3g
Posts: 16
Joined: Sun Mar 29, 2020 4:27 pm

Re: Project 13414

Post by Curt3g »

Thanks John. Picked up another 13414 this morning (Run 255, Clone 83, Gen 0). The estimated time between frames is 2 min 18 sec, so looks to be in the 4 hour range also.

Cheers,
Ichbin3
Posts: 96
Joined: Thu May 28, 2020 8:06 am
Hardware configuration: MSI H81M, G3240, RTX 2080Ti_Rev-A@220W, Ubuntu 18.04
Location: Germany

Re: Project 13414

Post by Ichbin3 »

I got a 13414 recently which was unexpected slow too
13414 (841,39,1)
TPF was 1:31 (other 13414 where 1:08 normaly) on a 2080Ti
Last edited by Ichbin3 on Tue Jun 23, 2020 4:38 am, edited 1 time in total.
Image
MSI H81M, G3240, RTX 2080Ti_Rev-A@220W, Ubuntu 18.04
kc2lrc
Posts: 21
Joined: Sun Jan 04, 2009 11:10 pm

Re: Project 13414

Post by kc2lrc »

I'm seeing some interesting performance numbers myself:

- 13415 on GTX 970, Debian 10: 1m 21s TPF, 106K points/day - nvidia-smi says GPU usage is around 46%
- 13415 on GTX 960, Windows 10: 1m 38s TPF, 79K points/day, HWMon shows GPU usage of 47%
- 13414 on GTX 1080, Debian 10: 2m 12s TPF, 1,076K points/day, nvidia-smi says GPU usage is 98%

Haven't seen it hit any of my 750Ti's yet.

Cheers -
Sam
Image
Ichbin3
Posts: 96
Joined: Thu May 28, 2020 8:06 am
Hardware configuration: MSI H81M, G3240, RTX 2080Ti_Rev-A@220W, Ubuntu 18.04
Location: Germany

Re: Project 13414

Post by Ichbin3 »

Just another slow one at my 2080Ti
13414 (1143,7,0)
TPF 1:46 vs 1:08 a normal one
Image
MSI H81M, G3240, RTX 2080Ti_Rev-A@220W, Ubuntu 18.04
cine.chris
Posts: 78
Joined: Sun Apr 26, 2020 1:29 pm

Re: Project 13414

Post by cine.chris »

I`ve seen a couple of very low PPD for WU 13414 today, which is why I`m here. Latest is on a 1660Ti, GPU utilization was at 75%.
Image Image
Joe_H
Site Admin
Posts: 7856
Joined: Tue Apr 21, 2009 4:41 pm
Hardware configuration: Mac Pro 2.8 quad 12 GB smp4
MacBook Pro 2.9 i7 8 GB smp2
Location: W. MA

Re: Project 13414

Post by Joe_H »

cine.chris wrote:I`ve seen a couple of very low PPD for WU 13414 today, which is why I`m here. Latest is on a 1660Ti, GPU utilization was at 75%.
What were the PRCG numbers for the underperforming WUs? They are trying to track down which Runs seem to have this issue and see if they can figure out why.
Image

iMac 2.8 i7 12 GB smp8, Mac Pro 2.8 quad 12 GB smp6
MacBook Pro 2.9 i7 8 GB smp3
kc2lrc
Posts: 21
Joined: Sun Jan 04, 2009 11:10 pm

Re: Project 13414

Post by kc2lrc »

My slugs were:

13415 (R 835, C 51, G 1) - 1m 21s on GTX 970 (Debian 10, Core 2 Duo E8200), reported 210ns/day, GPU usage around 47%
13415 (R 850, C 50, G 0) - 1m 23s on GTX 970, reported 207ns/day, wasn't around to see GPU usage
13415 (R 792, C 51, G 1) - 1m 40s on GTX 960 (Windows 10, i7-4790K), reported 128ns/day, GPU usage around 46%
13413 (R 767, C 3, G 0) - 2m 5s on GTX 960, reported 130ns/day, wasn't around to see GPU usage

No slugs are churning right now, but I'll keep my eyes open when I'm around.

Cheers -
Sam
Image
kc2lrc
Posts: 21
Joined: Sun Jan 04, 2009 11:10 pm

Re: Project 13414

Post by kc2lrc »

Noticed one more currently running:

13415 (1610, 8, 0) - 1m 44s on GTX 750Ti (Debian 10, Core 2 E8200), nvidia-smi reports GPU usage around 58%

Cheers -
Sam
Image
Curt3g
Posts: 16
Joined: Sun Mar 29, 2020 4:27 pm

Re: Project 13414

Post by Curt3g »

And a couple more today (RTX 2060), first looks as expected, second looks slow. And didn't mention in previous posts, running 24/7.

From earlier today: 13414 (Run 1816, Clone 4, Gen 1), TPF at 2 min 10 sec
Currently running: 13414 (Run 1553, Clone 12, Gen 0), TPF at 7 min, 18 sec

Regards,
Curt
uyaem
Posts: 222
Joined: Sat Mar 21, 2020 7:35 pm
Location: Esslingen, Germany

Re: Project 13414

Post by uyaem »

Adding to this another Run that seems to be slow (although with sample size = 1, I can't be sure if it's the entire Run):
Project 13414 (Run 879, Clone 26, Gen 0)

I've observed the same behaviour as others 75% GPU load (nVidia 1660 GTX).
Image
CPU: Ryzen 9 3900X (1x21 CPUs) ~ GPU: nVidia GeForce GTX 1660 Super (Asus)
ajm
Posts: 754
Joined: Sat Mar 21, 2020 5:22 am
Location: Lucerne, Switzerland

Re: Project 13414

Post by ajm »

Two slow ones more currently running:
13414 (1209, 20, 0) -> 1080 ti, TPF 3 mins 06 secs
13414 (1586, 20, 0) -> 2070 S, TPF 2 mins 40 secs

EDIT: one more, also currently running:
13414 (1618, 12, 0) -> 1050 ti, TPF 8 mins 51 secs

Three others currently running, but with a normal TPF:
13414 (2227, 14, 1) -> 1050 ti, TPF 6 mins 13 secs
13414 (1275, 23, 0) -> 5700 XT, TPF 2 mins 05 secs
13414 (2737, 22, 0) -> 2080 S, TPF 1 mins 22 secs
Post Reply