A) Task object, returned by submit call, has an argument finished which indicates status of the task:
Code: (python)
task = job_server.submit(f1, (a,b,c))Code: (python)
if task.finished:
print "The task is done!"
else
print "Still working on it ..."Q) I would like to perform an action at the time of completion for each individual task. Is it possible?
A) Yes, you need to setup callback argument of submit method, see PP callback example
Q) I do not want to run any tasks on my local computer. How do I do that?
A) ncpus argument of the pp.Server specifies the number of local ppworkers to use. If you do not specify this argument PP uses the numbers of ppworkers equal to the number of effective processors on your local host. If you set it to 0, three will be no local execution of tasks.
Q) How can I use pp.Template class in my program?
A) The purpose of Template class is to simplify submission of similar pp-tasks, which execute the same function with different arguments. There are two steps in using the Template class.
1) Creating a template which glues together a function with its dependencies and given instance of pp.Server:
Code: (python)
fn = pp.Template(job_server, sum_primes, (isprime,), ("math",))Code: (python)
job1 = fn.submit(100)
job2 = fn.submit(200)
Q) How to use PP on a PBS cluster?
A) PP can be used with PBS with or without autodiscovery.
Without autodiscovery you simply need to start ppserver.py on all nodes assigned to a job and then add all of them to a ppserver variable in the client Python application.
With autodiscovery you need to run ppserver.py with -a switch (./ppserver.py -a ) and then set ppservers = ("*",) in the client application.
In both cases it is useful to use -t switch to automatically kill all ppservers when client exits.
Q) What Python versions are supported?
A) PP was tested with Python 2.3 - 2.7 on a variety of platforms.
Python 3 port is available as well.
Q) How load balancing works in Parallel Python?
A) PP uses dynamic load balancing. Both local server and remote ppservers use ppworkers to execute jobs one at a time. Initially jobs are sent to remote ppservers according to the number of ppworkers they have. The number of remote jobs exceeds the number of ppworkers so that job input & output transfer could be interleaved with the computation. When execution of a remote job finishes, another job will be scheduled. As a result each host executes the number of jobs proportional to their computational capacity.
When both local and remote hosts are present, jobs are first scheduled on the localhost.
Q) What to do if large jobs execute correctly locally but I get "Socket connection is broken" when the run on remote ppserver?
A) Increase the socket timeout time (socket_timeout argument of the Server and -k command line argument of ppserver.py) to be an upper bound of your job execution time.
Q) Can Parallel Python work across different versions of Python?
A) Yes, the same version of Parallel Python works across different versions of Python as long as all of them are either Python 2.x or Python 3.x. For example Parallel Python will works on cluster that has Python 2.5, 2.6, 2.7 installed on various machines. However it won't work if ppclient is using Python 2.7 and ppserver is using Python 3.5.
By using this website, you signify your acceptance of Terms and Conditions and Privacy Policy.
© 2024 parallelpython.com All rights reserved
© 2024 parallelpython.com All rights reserved