Message boards : Number crunching : Problems and Technical Issues with Rosetta@home
Previous · 1 . . . 71 · 72 · 73 · 74 · 75 · 76 · 77 . . . 295 · Next
Author | Message |
---|---|
Mr P Hucker Send message Joined: 12 Aug 06 Posts: 1600 Credit: 11,300,065 RAC: 1,102 |
Regarding resource share settings......I have Rosetta at 1 and WCG at 9999, and yet Rosetta still takes control and suspends WCG tasks during this transition between machines. I am glad the BOINC client works for you 100% as intended. Which I am sure you have tested. Meanwhile I'll simply continue to complicate things. And so will I, as like you Boinc never does what I ask. You join Rosetta at 1 and it panics, thinking it's not done any over the last 10 days (WTF?) but it should do a 1/10000. So it runs it at 100%. Changing what projects you run and what the weighting is should reset the counter. I changed this on mine to make things slightly more sensible, in config.xml: <rec_half_life_days>1.000000</rec_half_life_days> - this means it looks at the last day instead of the last 10 days to figure out what to run. PS: Click on a person's name and it shows everything they have EVER done. It shows you with a recent credit on over a dozen. I guess it's an average over quite some time - I think it's a month. |
mikey Send message Joined: 5 Jan 06 Posts: 1895 Credit: 8,923,505 RAC: 755 |
Peter Huycker said It's a pity Boinc doesn't manage multiple computers and we have to use third party programs to do so. I use Boinctasks, and in fact I'd use it for a single machine too, because it's display is 10 times better than Boinc. For a start it colour codes running, queued, etc, and collapses a queue of 50 tasks into one line. The actual Boinc manager is unusable as an interface. I hope that's an option if it's ever implemented as I much prefer to manage each pc as it's own computer and then choose whether to run the same project as other computers or to run it's own set of projects. Sometimes I have every pc running the same project while other times I prefer to run something different on each pc. In some cases I just can't blast thru the tasks at a project because I am a 'Team friendly person' meaning if you are on my team and I am behind you I will not pass you as long as you are crunching. It's the old thing 'just because I can doesn't mean I should' for me!!! I have more resources than any otehr cruncher on my team and could be easily #1 on every running project I crunch for but that then ruins the incentive for my teammates to keep on crunching because they are #1 or #2. I am already #1 at enough projects that I can move things around to keep the pressure on but not pass them. ie at PrimeGrid a teammate is almost 30 million credits ahead of me but a new challenge could easily see me doing 5 million credits a week if I really want too. He just doesn't have, nor can he afford, the kind of horsepower needed to keep me behind him, but he keeps crunching the way he is and that's a good thing for Boinc in general!! |
Mr P Hucker Send message Joined: 12 Aug 06 Posts: 1600 Credit: 11,300,065 RAC: 1,102 |
Peter Huycker saidIt's a pity Boinc doesn't manage multiple computers and we have to use third party programs to do so. I use Boinctasks, and in fact I'd use it for a single machine too, because it's display is 10 times better than Boinc. For a start it colour codes running, queued, etc, and collapses a queue of 50 tasks into one line. The actual Boinc manager is unusable as an interface. You've never seen Boinctasks have you? You can do exactly what you want aswell as what I want. I can select one computer, a few, or all of them, and give them an instruction. I like you set different machines doing different things. Some are better at certain projects, some can't do them at all. |
Aravah Send message Joined: 12 Apr 20 Posts: 6 Credit: 1,101,172 RAC: 0 |
I am seeing Rosetta task requesting much much more memory than usual? Is this expected? Application Rosetta 4.20 Name kp8RjDVk_fold_and_dock_SAVE_ALL_OUT_1009390_4381 State Waiting for memory Received Thu 10 Sep 2020 16:00:19 BST Report deadline Sun 13 Sep 2020 16:00:18 BST Estimated computation size 80,000 GFLOPs CPU time 01:09:16 CPU time since checkpoint --- Elapsed time 01:17:21 Estimated time remaining 07:32:17 Fraction done 5.772% Virtual memory size 33.02 GB Working set size 28.44 GB Directory slots/6 Progress rate 4.320% per hour Executable rosetta_4.20_x86_64-pc-linux-gnu |
MarkJ Send message Joined: 28 Mar 20 Posts: 72 Credit: 25,238,680 RAC: 0 |
I am seeing Rosetta task requesting much much more memory than usual? Most of us had these earlier in the week. I aborted all the fold_and_dock tasks. They seem to have a serious problem with the amount of memory they need. BOINC blog |
Brian Nixon Send message Joined: 12 Apr 20 Posts: 293 Credit: 8,432,366 RAC: 0 |
Is this expected?Several users have reported fold_and_dock tasks trying (and usually eventually failing) to allocate tens of gigabytes of memory. If you have a vast amount of swap space they might be able to complete, but you’re probably as well just aborting them and doing something else instead. |
Grant (SSSF) Send message Joined: 28 Mar 20 Posts: 1636 Credit: 16,775,951 RAC: 9,341 |
I am seeing Rosetta task requesting much much more memory than usual?Because of the size of your cache, and Rosetta being your secondary project, it's taken you several days to start processing what was a resend of those problem Tasks. Grant Darwin NT |
Mr P Hucker Send message Joined: 12 Aug 06 Posts: 1600 Credit: 11,300,065 RAC: 1,102 |
Is this expected?Several users have reported fold_and_dock tasks trying (and usually eventually failing) to allocate tens of gigabytes of memory. If you have a vast amount of swap space they might be able to complete, but you’re probably as well just aborting them and doing something else instead. Don't most people have way more than 10s of GB of swap space? Most people have their OS set to the default of using whatever is required. One of my machines has 2TB free, two have 1TB free, and the other three have about 200GB free. I assume those tasks would have a shot at running on my machines. I don't know if I've had any, and may not have noticed. I only notice if something isn't using the processor very well (the CPU time and wall time are drastically different in Boinctasks), or if it's crashing (the task changes colour and shows "computation error"). |
Brian Nixon Send message Joined: 12 Apr 20 Posts: 293 Credit: 8,432,366 RAC: 0 |
What’s not clear is whether those tasks legitimately need all the memory they’re allocating (in which case they should run to completion on a system with sufficient resources) – or whether it’s a bug, and the tasks will simply keep requesting more memory (however much the system makes available) until they fail. I disable the paging file on every machine I operate. If that causes anything to fail to run, I need more RAM! |
robertmiles Send message Joined: 16 Jun 08 Posts: 1230 Credit: 14,172,067 RAC: 604 |
Is this expected?Several users have reported fold_and_dock tasks trying (and usually eventually failing) to allocate tens of gigabytes of memory. If you have a vast amount of swap space they might be able to complete, but you’re probably as well just aborting them and doing something else instead. You appear to be assuming that the available swap space is all of the unused space on the hard drive. I doubt if this is correct. However, I just tried checking my Windows 10 computer, and it appears that Windows 10 no longer uses anything called swap space. I remember that some previous versions of Windows had a reserved area on the hard drive known as swap space, |
Brian Nixon Send message Joined: 12 Apr 20 Posts: 293 Credit: 8,432,366 RAC: 0 |
‘Swap space’ is the name for disk-space-as-memory in UNIX-family operating systems. Windows calls it a ‘paging file’, and it’s still very much present in Windows 10. (Control Panel > System > Advanced system settings > Advanced tab > Performance group > Advanced tab > Virtual memory group) |
Mr P Hucker Send message Joined: 12 Aug 06 Posts: 1600 Credit: 11,300,065 RAC: 1,102 |
What’s not clear is whether those tasks legitimately need all the memory they’re allocating (in which case they should run to completion on a system with sufficient resources) – or whether it’s a bug, and the tasks will simply keep requesting more memory (however much the system makes available) until they fail. If you have an SSD, paging isn't as bad as it used to be with those rust spinners, where if it started being used, the interface became so slow you weren't sure if what you clicked was being done slowly or it hadn't noticed, so you click again then it ends up loading 5 of them slowing it down even further, and you spend so much time waiting for the computer you have no time to go to work to earn money to buy more RAM. |
Mr P Hucker Send message Joined: 12 Aug 06 Posts: 1600 Credit: 11,300,065 RAC: 1,102 |
Is this expected?Several users have reported fold_and_dock tasks trying (and usually eventually failing) to allocate tens of gigabytes of memory. If you have a vast amount of swap space they might be able to complete, but you’re probably as well just aborting them and doing something else instead. On this Windows 10 computer (and all of them unless you change it), paging is automatic. It will increase it if it needs it, but doesn't reserve a chunk of disk for it. |
robertmiles Send message Joined: 16 Jun 08 Posts: 1230 Credit: 14,172,067 RAC: 604 |
‘Swap space’ is the name for disk-space-as-memory in UNIX-family operating systems. Thanks. Windows 10 made it hard to find the right control panel if you haven't used it lately! Try Windows System / System and Security / System for the first few steps. About 4,8 GB currently on my computer. |
Mr P Hucker Send message Joined: 12 Aug 06 Posts: 1600 Credit: 11,300,065 RAC: 1,102 |
‘Swap space’ is the name for disk-space-as-memory in UNIX-family operating systems. Not sure why Microsoft has split all the controls up and put them in stupid places that only they think make sense. I often have to use the search to find the simplest of things. I want the old control panel with a sensible list. It's usually actually quicker to google the problem. That 4.8GB should automatically increase as required though. At least we never get "out of memory" messages like in very early Windows. |
Aravah Send message Joined: 12 Apr 20 Posts: 6 Credit: 1,101,172 RAC: 0 |
Thanks to all who replied to my previous message. I aborted that task and all has returned to normal :) Except I notice some tasks are awarded very few credits for the same Gflops as other tasks that are give an order of magnitude more. For example, just 35.12 credits for the first one but 326.58 for the second. It is to do with whether you're reporting the task result first or second? Name rb_09_10_37482_36690__t000__1_C1_SAVE_ALL_OUT_IGNORE_THE_REST_1010270_821_0 Workunit 1127882972 Created 12 Sep 2020, 0:02:14 UTC Sent 12 Sep 2020, 0:07:00 UTC Report deadline 15 Sep 2020, 0:07:00 UTC Received 14 Sep 2020, 4:25:44 UTC Server state Over Outcome Success Client state Done Exit status 0 (0x00000000) Computer ID 4466108 Run time 12 hours 55 min 41 sec CPU time 11 hours 4 min Validate state Valid Credit 35.12 Device peak FLOPS 3.09 GFLOPS Application version Rosetta v4.20 x86_64-pc-linux-gnu Peak working set size 1,135.22 MB Peak swap size 1,276.45 MB Peak disk usage 29.31 MB Name pdl1_graft_v1_SAVE_ALL_OUT_IGNORE_THE_REST_8ae9bz6m_1009438_2_0 Workunit 1127762369 Created 11 Sep 2020, 18:09:26 UTC Sent 11 Sep 2020, 19:34:11 UTC Report deadline 14 Sep 2020, 19:34:11 UTC Received 14 Sep 2020, 2:58:16 UTC Server state Over Outcome Success Client state Done Exit status 0 (0x00000000) Computer ID 4466108 Run time 11 hours 49 min 45 sec CPU time 9 hours 59 min 46 sec Validate state Valid Credit 326.58 Device peak FLOPS 3.09 GFLOPS Application version Rosetta v4.20 x86_64-pc-linux-gnu Peak working set size 1,010.59 MB Peak swap size 1,143.78 MB Peak disk usage 30.09 MB |
Brian Nixon Send message Joined: 12 Apr 20 Posts: 293 Credit: 8,432,366 RAC: 0 |
Rosetta tasks are almost never sent to more than one computer. The low credit one looks like a bug which crops up occasionally: somehow the task ran more than once, and only the last (very short) run received credit, even though it was the first run that did almost all the work. |
Mr P Hucker Send message Joined: 12 Aug 06 Posts: 1600 Credit: 11,300,065 RAC: 1,102 |
Rosetta tasks are almost never sent to more than one computer. The low credit one looks like a bug which crops up occasionally: somehow the task ran more than once, and only the last (very short) run received credit, even though it was the first run that did almost all the work. Is this just a credit bug? Is the data from the big first run still saved? |
Brian Nixon Send message Joined: 12 Apr 20 Posts: 293 Credit: 8,432,366 RAC: 0 |
Good question. I don’t know whether the results from the second run are added to, or overwrite, those from the first. |
Breno Send message Joined: 8 Apr 20 Posts: 30 Credit: 12,505,896 RAC: 4,608 |
I don't really know if this is a persistent issue. I noticed today that some finished foldit WUs stopped at 45,78% during results upload transfers, they allow re-attempts to send results, but doing so manually causes a very slow upload even w/o any upload transfer limits. Anyway, just thought team R@h should know. |
Message boards :
Number crunching :
Problems and Technical Issues with Rosetta@home
©2024 University of Washington
https://www.bakerlab.org