Unit using 1.5GB of RAM?

Message boards : Number crunching : Unit using 1.5GB of RAM?

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
sculptor

Send message
Joined: 25 Mar 09
Posts: 6
Credit: 2,604,245
RAC: 0
Message 67814 - Posted: 24 Sep 2010, 18:06:53 UTC

I have 2 PCs running Rosetta@Home, one with 2GB of RAM and the other has 4GB of RAM.
On the 2GB PC it is struggling running 2 WUs as one of the WUs is using 1.49940GB of RAM whilst the other WU is using 160,767KB.
I know WUs differ in the amount of RAM used but 1.5GB seems exsesive. The WU in question is:
Rosetta Mini 2.15 T0560_t2_rs_stg0_lrlxjcst_t000_casp9_SAVE_ALL_OUT_22170_821_1
Could you please advise as to wether this unit should be using this amount of RAM
Tom Alexander
ID: 67814 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mod.Sense
Volunteer moderator

Send message
Joined: 22 Aug 06
Posts: 4018
Credit: 0
RAC: 0
Message 67816 - Posted: 24 Sep 2010, 22:00:39 UTC

Yes, that seems pretty steep. What is the CPU time shown in the task manager? (i.e. how long since the last start did it take to consume that much memory?)

Rosetta Moderator: Mod.Sense
ID: 67816 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
sculptor

Send message
Joined: 25 Mar 09
Posts: 6
Credit: 2,604,245
RAC: 0
Message 67818 - Posted: 24 Sep 2010, 22:51:40 UTC

Its finished the WU, taking 3hrs 11mins.
As the PC was crunching another unit at the time, I suspended the unit and watched task manager to check Id stopped the 1.5GB one, so I knew the name of the WU.
When I restarted the unit the RAM use went up very quickly perhaps tens seconds to get to 1.4GB and another 10 seconds to get to 1.5GB.

On my 4GB PC it is running a unit with a very similar name but its only using 650,000k.

My main concern was that these WUs using a lot of RAM would become the norm, Ive ordered 4GB of RAM just incase, its a quad core rig so should perform better with 4GB instead of 2GB.
Tom Alexander.
ID: 67818 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
diederiks

Send message
Joined: 13 Oct 05
Posts: 2
Credit: 740,392
RAC: 0
Message 67819 - Posted: 24 Sep 2010, 22:53:04 UTC - in response to Message 67816.  

Yes, that seems pretty steep. What is the CPU time shown in the task manager? (i.e. how long since the last start did it take to consume that much memory?)

i have posted a simular problem in the rosetta v2.15 post, after first checkpoint was made my WU as using aroun 1080 MB of ram, now at 11% it's using +/-1160 MB
ID: 67819 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mod.Sense
Volunteer moderator

Send message
Joined: 22 Aug 06
Posts: 4018
Credit: 0
RAC: 0
Message 67823 - Posted: 25 Sep 2010, 17:07:34 UTC

There are so many protocols and proteins that it is difficult to define a norm. As the software evolves, there are often phases where memory usage increases. The Project Team is generally able to find modifications that lower it again. Reports like yours help them isolate the area of the program that is using so much more memory then normal.

The project guidelines have not changed for machine requirements.
Rosetta Moderator: Mod.Sense
ID: 67823 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Chilean
Avatar

Send message
Joined: 16 Oct 05
Posts: 711
Credit: 26,694,507
RAC: 0
Message 67824 - Posted: 25 Sep 2010, 18:28:46 UTC

Rosetta is welcome to use my memory. I still have like 2GB free running two threads of Rosetta...
ID: 67824 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jochen

Send message
Joined: 6 Jun 06
Posts: 133
Credit: 3,847,433
RAC: 0
Message 67826 - Posted: 25 Sep 2010, 21:45:42 UTC

The highest I've seen so far is as well 1.5 GB for a single WU.

This is giving me a headache too. Due to the 32-bit OS, I only got 3.4 GB of RAM available on my 4 + 4 core i7. It was already close to the edge with the 2.14's 500 to 600 MB WUs...

At the very moment, I'm typing this, one of the 1.5 GB tasks crashed... I'll post details in the appropriate thread tomorrow.
ID: 67826 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Chilean
Avatar

Send message
Joined: 16 Oct 05
Posts: 711
Credit: 26,694,507
RAC: 0
Message 67827 - Posted: 25 Sep 2010, 21:53:34 UTC - in response to Message 67826.  

The highest I've seen so far is as well 1.5 GB for a single WU.

This is giving me a headache too. Due to the 32-bit OS, I only got 3.4 GB of RAM available on my 4 + 4 core i7. It was already close to the edge with the 2.14's 500 to 600 MB WUs...

At the very moment, I'm typing this, one of the 1.5 GB tasks crashed... I'll post details in the appropriate thread tomorrow.


Switch to 64-bit! =]
ID: 67827 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Roland

Send message
Joined: 31 Mar 09
Posts: 5
Credit: 462,050
RAC: 0
Message 67830 - Posted: 26 Sep 2010, 0:10:17 UTC

Right now i'm using 1.641 gb for one unit. It is bit of waste of processor power since I have 4 cores but only using 2 since one takes so much memory. not sure if it is the mem_widd_run02_Mevn_a_2kdc_SAVE_ALL_OUT_IGNORE_THE_REST_221578214_0 or T0536........, but i think the first one
ID: 67830 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jochen

Send message
Joined: 6 Jun 06
Posts: 133
Credit: 3,847,433
RAC: 0
Message 67833 - Posted: 26 Sep 2010, 9:27:14 UTC - in response to Message 67827.  

Switch to 64-bit! =]


I can't. That one is my computer at the office and I installed XP 64 in the first place, after assembling this computer. But some applications didn't run properly, so I had to revert to 32 bit.

None of my 3 computers is actually up to tasks using 1.5 GB of RAM. Even my 12 cores machine has 'only' 12 GB of RAM, which is at this point already 4 GB short.

The new memory requirements for Rosetta are:
2 cores: 3 GB
4 cores: 6 GB
6 cores: 8 GB
8 cores: 12 GB
12 cores: 16 GB

:/
ID: 67833 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Chris Holvenstot
Avatar

Send message
Joined: 2 May 10
Posts: 220
Credit: 9,106,918
RAC: 0
Message 67836 - Posted: 26 Sep 2010, 13:16:41 UTC

Jochen previously stated ...

The new memory requirements for Rosetta are:
2 cores: 3 GB
4 cores: 6 GB
6 cores: 8 GB
8 cores: 12 GB
12 cores: 16 GB


Jochen, where are these new requirements stated? I just took a gander at the systems requirements page and it still states 512 meg. Is this an official requirement or just an extrapolation from the reported 1.5 gig for a few tasks?

Damn, when I assembled my systems I allocated 1 gig per core for my Rosetta crunchers - if this is truly what is going to be expected going into the future, and it is a "real memory" requirement verses swap space, its going to put me out of the game for a while ...

CH

ID: 67836 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jochen

Send message
Joined: 6 Jun 06
Posts: 133
Credit: 3,847,433
RAC: 0
Message 67837 - Posted: 26 Sep 2010, 16:41:04 UTC
Last modified: 26 Sep 2010, 16:42:44 UTC

Hello Chris,

these are just my observations. And as well it's a 'worst case' scenario. But since there are WUs utilizing 1.5 GB of RAM, one could end up running only these WUs.
I already ran out of Memory on my two 32-bit Systems, resulting in idling cores. I never ran out of memory with 2.14 tasks...

Currently, there are 4 WUs on my 6 + 6 core machine, each utilizing 1.3 to 1.5 GB. And another two are utilizing 670 MB and 850 MB respectively. The overall memory utilization is already at 88% of the available 12 GB RAM.

cu

Joe
ID: 67837 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jochen

Send message
Joined: 6 Jun 06
Posts: 133
Credit: 3,847,433
RAC: 0
Message 67841 - Posted: 27 Sep 2010, 15:32:21 UTC

It's turning nasty... On my 'big' computer, I had two tasks 'waiting for memory', thus two cores idling. As I assumed before, 1 GB per core is no longer enough.

Of course they won't change the system requirements page. These are the minimum requirements, not the recommended or optimum ones.

I'm just wondering, what happens, if somebody with only 512 MB of RAM gets on of those 1.8 GB tasks, I've seen?


ID: 67841 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mod.Sense
Volunteer moderator

Send message
Joined: 22 Aug 06
Posts: 4018
Credit: 0
RAC: 0
Message 67842 - Posted: 27 Sep 2010, 15:41:10 UTC

The requirements have not changed. The current mix of tasks seem to include more that require high amounts of memory. The mix will vary over time, and hopefully down the road they will find some ways of running the same work within a smaller memory footprint.

Until then, you might consider keeping a slightly larger cache of work on-hand. When a core goes idle due to "waiting for memory", BOINC will fire up another thread and run it as long as it can. It may run in less memory, or perhaps by the time it requires more, one of the other tasks has released sufficient memory. Either way, a larger list of tasks to be processing improves your odds that no cores go idle.

R@h has two types of work units. Regular, and "high-memory". These high-memory tasks are not dispatched to machines with only the minimum memory. This helps the tasks run well, however the memory usage you are all reporting is still going to cause swapping and less then smooth running for a number of hosts out there. It will need to be addressed.
Rosetta Moderator: Mod.Sense
ID: 67842 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Chilean
Avatar

Send message
Joined: 16 Oct 05
Posts: 711
Credit: 26,694,507
RAC: 0
Message 67843 - Posted: 27 Sep 2010, 17:24:58 UTC - in response to Message 67841.  

It's turning nasty... On my 'big' computer, I had two tasks 'waiting for memory', thus two cores idling. As I assumed before, 1 GB per core is no longer enough.

Of course they won't change the system requirements page. These are the minimum requirements, not the recommended or optimum ones.

I'm just wondering, what happens, if somebody with only 512 MB of RAM gets on of those 1.8 GB tasks, I've seen?



What are your settings?
Because if you set your "allowed RAM usage" by BOINC too low, then you'll have WUs waiting for memory irregardless of how much RAM you actually have installed.
ID: 67843 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jochen

Send message
Joined: 6 Jun 06
Posts: 133
Credit: 3,847,433
RAC: 0
Message 67844 - Posted: 27 Sep 2010, 17:55:29 UTC - in response to Message 67842.  
Last modified: 27 Sep 2010, 18:05:36 UTC

Mod.Sense said:
When a core goes idle due to "waiting for memory", BOINC will fire up another thread and run it as long as it can. It may run in less memory, or perhaps by the time it requires more, one of the other tasks has released sufficient memory. Either way, a larger list of tasks to be processing improves your odds that no cores go idle.


This is simply not true. There was and still is a sufficent number of WUs available, but no new task is started when a task is automatically suspended and waits for memory. The only way to start another task is to manually suspend the tasks that are waiting for memory (as of BOINC 6.10.56).
ID: 67844 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jochen

Send message
Joined: 6 Jun 06
Posts: 133
Credit: 3,847,433
RAC: 0
Message 67845 - Posted: 27 Sep 2010, 18:04:28 UTC - in response to Message 67843.  

The Chilean said:
What are your settings?
Because if you set your "allowed RAM usage" by BOINC too low, then you'll have WUs waiting for memory irregardless of how much RAM you actually have installed.


memory usage (idle and load): 95%
swap file usage to 90%.


ID: 67845 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
M

Send message
Joined: 24 Oct 07
Posts: 5
Credit: 119,215
RAC: 0
Message 67861 - Posted: 29 Sep 2010, 7:05:12 UTC

Hmm... with these new gigabyte size WU's I'm having to scale back my 6gig i7 from 8 wu's to 4 wu's simultaniously.

I've had repeated memory warnings & crashes.

M.
ID: 67861 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mikey
Avatar

Send message
Joined: 5 Jan 06
Posts: 1895
Credit: 9,135,082
RAC: 4,703
Message 67868 - Posted: 29 Sep 2010, 11:25:55 UTC - in response to Message 67861.  

Hmm... with these new gigabyte size WU's I'm having to scale back my 6gig i7 from 8 wu's to 4 wu's simultaniously.

I've had repeated memory warnings & crashes.

M.


Yeah I am thinking of upping the ram in one machine to 4gig just so it can run 2 units at once on its dual core cpu, the 2nd pc already has 4 gig of ram in it.
ID: 67868 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jochen

Send message
Joined: 6 Jun 06
Posts: 133
Credit: 3,847,433
RAC: 0
Message 67874 - Posted: 29 Sep 2010, 12:32:04 UTC

Still 1.8 GB per task is just ridiculous. I moved over to POEM.

Send me an email, once this has been fixed and I might return...

Live long and prosper,

Joe
ID: 67874 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
1 · 2 · Next

Message boards : Number crunching : Unit using 1.5GB of RAM?



©2024 University of Washington
https://www.bakerlab.org