Mark and UNIX computer cluster

announcements (new versions, changes, bugs, installation problems...) related to program MARK

Mark and UNIX computer cluster

Postby kris_deer » Thu Mar 20, 2014 5:01 pm

Hi there,

To save up time, I am using Mark for Linux (version ) and RMark in command batch on a computer cluster (Grid Engine) - doing so let me adjut several models as a same time and free my own machine. Note that I am not trying to use 'real' parallel computing within one job, just running several instances of Mark for the different jobs (model fit).

Everything has been working like a charm until today. The sys admin killed all my jobs, pointing out that each running job uses between 1500% and 2000% CPU (~ 15 cores) - usually 1 one job means 1 core. Actually, Mark behaves like it was able to use all available cores for each job on given cluster. I could perfectly live with this myself but the admin does not see it that way - at all.

So here I am. Has anyone experienced a similar 'problem'? Is it a feature of Mark? Any tips on how to handle this problem and force Mark to use 1 core at a time would be greatly appreciated. On my local linux box (Debian Jessie), Mark is always confined to one core and works as I expect. I apologize if you guys think this is not related to Mark but running purely R jobs do not lead to the same issue though.

Thanks

Kris
kris_deer
 
Posts: 2
Joined: Thu Mar 20, 2014 11:16 am

Re: Mark and UNIX computer cluster

Postby jlaake » Thu Mar 20, 2014 5:09 pm

Ouch. At least they could have given you a warning. By default MARK.exe uses parallel computing and will take #cpus-1 to do so. You can play nice by setting the threads argument for mark or mark.wrapper to some smaller value such that your sys.admin won't kill your jobs.

--jeff
jlaake
 
Posts: 1417
Joined: Fri May 12, 2006 12:50 pm
Location: Escondido, CA

Re: Mark and UNIX computer cluster

Postby kris_deer » Thu Mar 20, 2014 5:22 pm

Thank you Jeff for this swift answer. This is explicit in mark R help, indeed (RT*M). Will set this thread argument to 2 or even 3 on the cluster :evil: ... You guys are doing a fantastic job! The last Linux version rocks.

Kris
kris_deer
 
Posts: 2
Joined: Thu Mar 20, 2014 11:16 am


Return to software problems/news

Who is online

Users browsing this forum: No registered users and 4 guests