Message boards : Number crunching : How might memory effect R@h processing?
Author | Message |
---|---|
Stephen "Heretic" Send message Joined: 2 Apr 20 Posts: 21 Credit: 11,028 RAC: 0 |
. . I finally have some completed tasks and I delved into the result files for information. . . The first two results had the following errors ...
Reason: Access Violation (0xc0000005) at address 0x019F2BE0 write attempt to address 0x017D7EC9 - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x01AC2BE0 write attempt to address 0x017D7EC9
|
MarkJ Send message Joined: 28 Mar 20 Posts: 72 Credit: 25,238,680 RAC: 0 |
. . I finally have some completed tasks and I delved into the result files for information. You’d better run memtest86+ on it. If it’s overclocked then reset to factory defaults. I had one machine that did that last week. Never had a problem with it before. I ran memtest86+ on it. Narrowed it down to a pair of memory sticks (the i7 in question had 4 x 8GB sticks). I’ve since replaced all 4 sticks and sent the dud pair back under warranty. BOINC blog |
Stephen "Heretic" Send message Joined: 2 Apr 20 Posts: 21 Credit: 11,028 RAC: 0 |
You’d better run memtest86+ on it. If it’s overclocked then reset to factory defaults. . . I never overclock, I'm not that adventurous. But it sounds like a memtest session might be a good idea. If it fails I might have to fork out some dollars for a pair of 8GB sticks, it is running just one x 8GB at the moment. . . Thanks Stephen . . |
Grant (SSSF) Send message Joined: 28 Mar 20 Posts: 1681 Credit: 17,854,150 RAC: 20,118 |
it is running just one x 8GB at the moment.That would slow the whole system down quite a bit with anything requiring access to system memory as it wouldn't be able to use Dual Channel mode with only the 1 module. Dual channel systems, always use modules in pairs, Quad channel, always use them in groups of 4, etc. Grant Darwin NT |
Stephen "Heretic" Send message Joined: 2 Apr 20 Posts: 21 Credit: 11,028 RAC: 0 |
. . Thanks Grant but I did know that. This system is using a low end MSI MoB with only 2 memory slots, to be honest I would have to do research to even be sure it can do dual channel mode, though I expect it does. But if I have to replace the one stick it came with I would go all out and get a pair. Stephen :) |
robertmiles Send message Joined: 16 Jun 08 Posts: 1232 Credit: 14,281,662 RAC: 1,548 |
You’d better run memtest86+ on it. If it’s overclocked then reset to factory defaults. Having two memory sticks instead of just one tends to increase the speed of programs somewhat. Even more with four, if your motherboard can handle four properly. |
Klimax Send message Joined: 27 Apr 07 Posts: 44 Credit: 2,800,788 RAC: 736 |
[snip] Resource contention, nothing less, nothing more. Nothing exotic or unknowable. ZEN architecture is strong on memory bandwidth but bad on memory latency. Even if you have relatively larger cache there will be still lots of memory access that has to be fulfilled from main memory. Because of that too many core doing access will overload memory bus and their performance will drop of sharply. (Especially when latency of memory access is already high) I suggest you experiment with number of concurrent task for RAH to find equilibrium. Alternately, you could try to see if you can get memory to higher frequency without worsening its timing. Also ensure that there is no swapping. (IIRC each 4.12 task needs about 2GB of RAM which amounts to about 128GB of RAM in use beside all other processes already running) |
Jim1348 Send message Joined: 19 Jan 06 Posts: 881 Credit: 52,257,545 RAC: 0 |
4.12 is running very poorly on AMD. I get in on the middle of this discussion, but how are you measuring that? My Ryzens (2600, 3600, 3700X) are running fine. The scores on 4.12 are at least as good as 4.07 and 4.08. And on the Ryzen 3600, the stderr_txt shows exactly the same "Device peak FLOPS 5.95 GFLOPS" for both 4.07 and 4.12, for example. For the Ryzen 2600, it is 5.05 GFLOPS in both cases. It is about the same on my i7-8700. Whatever the problem, I don't think it is due to the Ryzens. EDIT: I have all machines running all cores on Ryzens, reserving one for a GPU (at least until the recent shortage; most of them still are). So if it were a loading effect, it would show up. EDIT 2: Your Threadripper handles cache differently than the Ryzens. That may have something to do with what you are seeing. |
strongboes Send message Joined: 3 Mar 20 Posts: 27 Credit: 5,394,270 RAC: 0 |
I'm making the opinion from a number of factors, but a hard figure is the average processing rate which is found on the computer details/application details page. 4.07 was regularly hitting 30 on the covid design tasks, 4.12 and 11.5 is the limit. 4.07 I was averaging 1 credit per core, per average 11.5 seconds compute time. 4.12 is running at something like 40-50 seconds average for 1 credit, the best I've noted was around 34 secs. I don't know if it's an accurate number to go on because you can run 100 tasks with the same name, 30 of them will run 10 decoys (for example) finish within 0.5% of time of each other, and the credit awarded can vary by 300 points, some will get 50, others rarely nearly 400. I'm assuming most people don't pay as much attention to these numbers as I have done and so haven't noticed. |
Mod.Sense Volunteer moderator Send message Joined: 22 Aug 06 Posts: 4018 Credit: 0 RAC: 0 |
I suspect there are tasks that are hitting the watchdog timeout (I seem to see a few every time I look at someone's hosts and WUs), and that this may explain poor credit/hour reports. The irregular granted credit then rolls up in to your other summary numbers. So, without something more directly comparable, I wouldn't take these as symptoms specific to any particular CPU type. Rosetta Moderator: Mod.Sense |
Keith Myers Send message Joined: 29 Mar 20 Posts: 97 Credit: 332,619 RAC: 244 |
It is a problem on my 3950X and 2920X AMD processors. I barely cracked 3.0 GFLOPS on my cpus with 12 cores running. The 4.07 app was faster than the 4.08 app. I would have loved if the performance had reached the 5 GFLOPS rating you are getting on your Ryzen 7's. |
Jim1348 Send message Joined: 19 Jan 06 Posts: 881 Credit: 52,257,545 RAC: 0 |
It is a problem on my 3950X and 2920X AMD processors. I barely cracked 3.0 GFLOPS on my cpus with 12 cores running. Interesting. I just happen to have a new Ryzen 3950X which I use on Folding as of yesterday. It is doing great, with around 550 k PPD on the new Covid-19s (Ubuntu 18.04.4). But it has only 16 GB of memory (though fast 3200 MHz 14-14-14-31), and so it is not suitable for Rosetta. I guess I lucked out on that one. EDIT: That is on 30 out of 32 threads. I reserve two for a GTX 1070, which adds another 850 k PPD. |
Keith Myers Send message Joined: 29 Mar 20 Posts: 97 Credit: 332,619 RAC: 244 |
It is a problem on my 3950X and 2920X AMD processors. I barely cracked 3.0 GFLOPS on my cpus with 12 cores running. I froze the computer up solid through 3 reboots once I added Rosetta. Every time that BOINC started the computer froze with the disk access light solid. I didn't realize that the 22 cpu tasks I ran for SETI in %cpu usage was going to try and run 22 Rosetta tasks with just 16GB of memory installed. I finally was able to stop BOINC from loading so I could manually edit the global and override preferences files to knock the number of cpu threads down to a value where the Rosetta startup wouldn't consume all my memory. I eventually stole memory from another SETI mothballed computer to add another 16GB to the computer I was running Rosetta on. |
Stephen "Heretic" Send message Joined: 2 Apr 20 Posts: 21 Credit: 11,028 RAC: 0 |
I froze the computer up solid through 3 reboots once I added Rosetta. Every time that BOINC started the computer froze with the disk access light solid. I didn't realize that the 22 cpu tasks I ran for SETI in %cpu usage was going to try and run 22 Rosetta tasks with just 16GB of memory installed. . . That is exactly the problem I have been suffering. I would like to run 2 x Rosetta tasks on my i5-6400 and support the GTX1050 running E@H but I only have 8 GB Ram. It is OK with the 32 bit tasks, I have 2 CPU cores committed to Rosetta and it mainly runs just one task, but sometimes there must be enough memory free and it will run 2. The good thing is that when memory demands increase it simply suspends one of the running Rosetta tasks and keeps on ticking, no harm, no foul. But when I have 64 bit tasks it goes to hell in a handbasket. At the moment I have it running OK by suspending the other 64bit Rosetta tasks so that they do NOT attempt to run. It works just fine like that but I have to check the machine more often than I should need to. I would love to know just how much ram is needed to keep 2 64 bit Rosetta tasks happy and still support the GPU. This MoBo only has 2 memory slots so I cannot go past 16GB even if I decide to invest more dollars in this project. I never had this problem running S@H. It ran 3 + 1 as smooth as silk. So in the end, with 32Gb Ram were you able to support all 22 tasks it was trying to run? Or did have to restrict it to any particular number? Stephen ? ? |
Grant (SSSF) Send message Joined: 28 Mar 20 Posts: 1681 Credit: 17,854,150 RAC: 20,118 |
At the moment I have it running OK by suspending the other 64bit Rosetta tasks so that they do NOT attempt to run. It works just fine like that but I have to check the machine more often than I should need to. I would love to know just how much ram is needed to keep 2 64 bit Rosetta tasks happy and still support the GPU.On my system, of 2 Tasks being processed by Rosetta v4.12 windows_x86_64, one is using 750MB of RAM, the other 320MB of RAM. It's similar to CPU & GPU work on Seti- there are no CPU or GPU tasks, just tasks. Same with the Rosetta v4.12 windows_intelx86 and Rosetta v4.12 windows_x86_64 applications. The Task is allocated to the Application when it is downloaded. The amount of required RAM depends on the Task, not the application being used. This MoBo only has 2 memory slots so I cannot go past 16GB even if I decide to invest more dollars in this project.It doesn't support 16GB DIMMs? Grant Darwin NT |
Stephen "Heretic" Send message Joined: 2 Apr 20 Posts: 21 Credit: 11,028 RAC: 0 |
I would love to know just how much ram is needed to keep 2 64 bit Rosetta tasks happy and still support the GPU. . . Well at those levels I should be AOK as is, the E@H tasks do not require more than 2GB of system ram so there is 5 GB or more left for Rosetta. But it aint happening. How are you seeing how much system ram each task is using? It's similar to CPU & GPU work on Seti- there are no CPU or GPU tasks, just tasks. Same with the Rosetta v4.12 windows_intelx86 and Rosetta v4.12 windows_x86_64 applications. The Task is allocated to the Application when it is downloaded. . . Thanks Grant, but I did know that. The amount of required RAM depends on the Task, not the application being used. . . That may be so but it does not fit the pattern I am seeing, so if it is not a memory insufficiency then there is something about the 64 bit app that does NOT like my machine. I need a way to preclude the 64 bit app from running to confirm this over a larger sample of tasks. This MoBo only has 2 memory slots so I cannot go past 16GB even if I decide to invest more dollars in this project.It doesn't support 16GB DIMMs? . . It is an old cheapy MSI MoBo. I have very little info but I am pretty sure when I bought it the max was 16GB. Stephen :( |
Grant (SSSF) Send message Joined: 28 Mar 20 Posts: 1681 Credit: 17,854,150 RAC: 20,118 |
How are you seeing how much system ram each task is using?Task Manager, Processes tab, Memory column. It can vary by several hundred MB depending on where in the Task it is at. Although Process Explorer is probably better as it shows how much is also reserved, but not using at the time- it's values are higher than those shown in Task Manager. A Task using 730MB in Task Manager shows as 760MB Private/ 783MB Working Set in Process Explorer. Check the motherboard manual, it should list what capacity & speed DIMMs are supported.. . It is an old cheapy MSI MoBo. I have very little info but I am pretty sure when I bought it the max was 16GB.This MoBo only has 2 memory slots so I cannot go past 16GB even if I decide to invest more dollars in this project.It doesn't support 16GB DIMMs? Grant Darwin NT |
Tom M Send message Joined: 20 Jun 17 Posts: 87 Credit: 15,166,437 RAC: 39,077 |
How are you seeing how much system ram each task is using?Task Manager, Processes tab, Memory column. Some of the ram merchants have a ram tester that will recommend the highest level of ram that a MB will support. If you don't have a lot of cpu cores/threads running Rosetti@Home you should be able to run multiples of the task even if it is running up near 2GB per task on a 16GB ram MB. Lately mine have all been a lot smaller. Tom Help, my tagline is missing..... Help, my tagline is......... Help, m........ Hel..... |
Keith Myers Send message Joined: 29 Mar 20 Posts: 97 Credit: 332,619 RAC: 244 |
So in the end, with 32Gb Ram were you able to support all 22 tasks it was trying to run? Or did have to restrict it to any particular number? I knocked it down to 16 x86_64 cpu tasks so I could run the three gpu tasks. If I tried to run more, I knocked gpu tasks offline with out of memory-postponed messages. That was with use 90% of memory while idle, 85% while active preferences. Was consuming a little shy of 30GB. The host is my daily driver. |
robertmiles Send message Joined: 16 Jun 08 Posts: 1232 Credit: 14,281,662 RAC: 1,548 |
I froze the computer up solid through 3 reboots once I added Rosetta. Every time that BOINC started the computer froze with the disk access light solid. I didn't realize that the 22 cpu tasks I ran for SETI in %cpu usage was going to try and run 22 Rosetta tasks with just 16GB of memory installed. The Rosetta tasks seem to run best with 2 GB per task, plus more for the CPU portion of any GPU task, plus more for the operating system. For 8 GB, two Rosetta tasks running at once is about the limit. If you have enough virtual cores, 16 GB should raise the limit to about 6 tasks. This assumes that you don't let any BOINC tasks stay in memory while they are not running. Expect lower numbers if you do. Filling both slots with the same model of memory board gives a little more speed than mismatched memory boards. You might also check if you can get faster memory boards that are compatible with your computer. Some models of computers are compatible with memory boards that contain more memory. Back when I first started with BOINC, I used this site to find compatible types of memory: https://www.crucial.com/store/systemscanner I now order any new computers with the largest and fastest memories that are compatible with that model. |
Message boards :
Number crunching :
How might memory effect R@h processing?
©2024 University of Washington
https://www.bakerlab.org